the plan for the future
As on 19 September 2013
Taken from: Wikipedia - Confirmation bias
Confirmation bias (also called confirmatory bias or myside bias) is a tendency of people to favor information that confirms their beliefs or hypotheses.[Note 1] People display this bias when they gather or remember information selectively, or when they interpret it in a biased way. The effect is stronger for emotionally charged issues and for deeply entrenched beliefs. They also tend to interpret ambiguous evidence as supporting their existing position. Biased search, interpretation and memory have been invoked to explain attitude polarization (when a disagreement becomes more extreme even though the different parties are exposed to the same evidence), belief perseverance (when beliefs persist after the evidence for them is shown to be false), the irrational primacy effect (a greater reliance on information encountered early in a series) and illusory correlation (when people falsely perceive an association between two events or situations).
A series of experiments in the 1960s suggested that people are biased toward confirming their existing beliefs. Later work re-interpreted these results as a tendency to test ideas in a one-sided way, focusing on one possibility and ignoring alternatives. In certain situations, this tendency can bias people's conclusions. Explanations for the observed biases include wishful thinking and the limited human capacity to process information. Another explanation is that people show confirmation bias because they are weighing up the costs of being wrong, rather than investigating in a neutral, scientific way.
Confirmation biases contribute to overconfidence in personal beliefs and can maintain or strengthen beliefs in the face of contrary evidence. Poor decisions due to these biases have been found in political and organizational contexts.
Confirmation biases are effects in information processing, distinct from the behavioral confirmation effect, also called "self-fulfilling prophecy", in which people's expectations affect their behaviour to make the expectations come true. Some psychologists use "confirmation bias" to refer to any way in which people avoid rejecting a belief, whether in searching for evidence, interpreting it, or recalling it from memory. Others restrict the term to selective collection of evidence.[Note 2]
Biased search for informationExperiments have repeatedly found that people tend to test hypotheses in a one-sided way, by searching for evidence consistent with the hypothesis they hold at a given time. Rather than searching through all the relevant evidence, they ask questions that are phrased so that an affirmative answer supports their hypothesis. They look for the consequences that they would expect if their hypothesis were true, rather than what would happen if it were false. For example, someone who is trying to identify a number using yes/no questions and suspects that the number is 3 might ask, "Is it an odd number?" People prefer this sort of question, called a "positive test", even when a negative test such as "Is it an even number?" would yield exactly the same information. However, this does not mean that people seek tests that are guaranteed to give a positive answer. In studies where subjects could select either such pseudo-tests or genuinely diagnostic ones, they favored the genuinely diagnostic.
The preference for positive tests is not itself a bias, since positive tests can be highly informative. However, in conjunction with other effects, this strategy can confirm existing beliefs or assumptions, independently of whether they are true. In real-world situations, evidence is often complex and mixed. For example, various contradictory ideas about someone could each be supported by concentrating on one aspect of his or her behavior. Thus any search for evidence in favor of a hypothesis is likely to succeed. One illustration of this is the way the phrasing of a question can significantly change the answer. For example, people who are asked, "Are you happy with your social life?" report greater satisfaction than those asked, "Are you unhappy with your social life?"
Even a small change in the wording of a question can affect how people search through available information, and hence the conclusions they reach. This was shown using a fictional child custody case. Subjects read that Parent A was moderately suitable to be the guardian in multiple ways. Parent B had a mix of salient positive and negative qualities: a close relationship with the child but a job that would take him or her away for long periods. When asked, "Which parent should have custody of the child?" the subjects looked for positive attributes and a majority chose Parent B. However, when the question was, "Which parent should be denied custody of the child?" they looked for negative attributes, but again a majority answered Parent B, implying that Parent A should have custody.
Similar studies have demonstrated how people engage in biased search for information, but also that this phenomenon may be limited by a preference for genuine diagnostic tests, where they are available. In an initial experiment, subjects had to rate another person on the introversion–extroversion personality dimension on the basis of an interview. They chose the interview questions from a given list. When the interviewee was introduced as an introvert, the subjects chose questions that presumed introversion, such as, "What do you find unpleasant about noisy parties?" When the interviewee was described as extroverted, almost all the questions presumed extroversion, such as, "What would you do to liven up a dull party?" These loaded questions gave the interviewees little or no opportunity to falsify the hypothesis about them. However, a later version of the experiment gave the subjects less presumptive questions to choose from, such as, "Do you shy away from social interactions?" Subjects preferred to ask these more diagnostic questions, showing only a weak bias towards positive tests. This pattern, of a main preference for diagnostic tests and a weaker preference for positive tests, has been replicated in other studies.
Another experiment gave subjects a particularly complex rule-discovery task involving moving objects simulated by a computer. Objects on the computer screen followed specific laws, which the subjects had to figure out. They could "fire" objects across the screen to test their hypotheses. Despite making many attempts over a ten-hour session, none of the subjects worked out the rules of the system. They typically sought to confirm rather than falsify their hypotheses, and were reluctant to consider alternatives. Even after seeing evidence that objectively refuted their working hypotheses, they frequently continued doing the same tests. Some of the subjects were instructed in proper hypothesis-testing, but these instructions had almost no effect.
Biased interpretationConfirmation biases are not limited to the collection of evidence. Even if two individuals have the same information, the way they interpret it can be biased.
A team at Stanford University ran an experiment with subjects who felt strongly about capital punishment, with half in favor and half against. Each of these subjects read descriptions of two studies; a comparison of U.S. states with and without the death penalty, and a comparison of murder rates in a state before and after the introduction of the death penalty. After reading a quick description of each study, the subjects were asked whether their opinions had changed. They then read a much more detailed account of each study's procedure and had to rate how well-conducted and convincing that research was. In fact, the studies were fictional. Half the subjects were told that one kind of study supported the deterrent effect and the other undermined it, while for other subjects the conclusions were swapped.
The subjects, whether proponents or opponents, reported shifting their attitudes slightly in the direction of the first study they read. Once they read the more detailed descriptions of the two studies, they almost all returned to their original belief regardless of the evidence provided, pointing to details that supported their viewpoint and disregarding anything contrary. Subjects described studies supporting their pre-existing view as superior to those that contradicted it, in detailed and specific ways. Writing about a study that seemed to undermine the deterrence effect, a death penalty proponent wrote, "The research didn't cover a long enough period of time", while an opponent's comment on the same study said, "No strong evidence to contradict the researchers has been presented". The results illustrated that people set higher standards of evidence for hypotheses that go against their current expectations. This effect, known as "disconfirmation bias", has been supported by other experiments.
An MRI scanner allowed researchers to examine how the human brain deals with unwelcome information.A study of biased interpretation took place during the 2004 US presidential election and involved subjects who described themselves as having strong feelings about the candidates. They were shown apparently contradictory pairs of statements, either from Republican candidate George W. Bush, Democratic candidate John Kerry or a politically neutral public figure. They were also given further statements that made the apparent contradiction seem reasonable. From these three pieces of information, they had to decide whether or not each individual's statements were inconsistent. There were strong differences in these evaluations, with subjects much more likely to interpret statements by the candidate they opposed as contradictory.
In this experiment, the subjects made their judgments while in a magnetic resonance imaging (MRI) scanner which monitored their brain activity. As subjects evaluated contradictory statements by their favored candidate, emotional centers of their brains were aroused. This did not happen with the statements by the other figures. The experimenters inferred that the different responses to the statements were not due to passive reasoning errors. Instead, the subjects were actively reducing the cognitive dissonance induced by reading about their favored candidate's irrational or hypocritical behavior.
Biased interpretation is not restricted to emotionally significant topics. In another experiment, subjects were told a story about a theft. They had to rate the evidential importance of statements arguing either for or against a particular character being responsible. When they hypothesized that character's guilt, they rated statements supporting that hypothesis as more important than conflicting statements.
Biased memoryEven if people have sought and interpreted evidence in a neutral manner, they may still remember it selectively to reinforce their expectations. This effect is called "selective recall", "confirmatory memory" or "access-biased memory". Psychological theories differ in their predictions about selective recall. Schema theory predicts that information matching prior expectations will be more easily stored and recalled. Some alternative approaches say that surprising information stands out more and so is more memorable. Predictions from both these theories have been confirmed in different experimental contexts, with no theory winning outright.
In one study, subjects read a profile of a woman which described a mix of introverted and extroverted behaviors. They later had to recall examples of her introversion and extroversion. One group was told this was to assess the woman for a job as a librarian, while a second group were told it was for a job in real estate sales. There was a significant difference between what these two groups recalled, with the "librarian" group recalling more examples of introversion and the "sales" groups recalling more extroverted behavior. A selective memory effect has also been shown in experiments that manipulate the desirability of personality types. In one of these, a group of subjects were shown evidence that extroverted people are more successful than introverts. Another group were told the opposite. In a subsequent, apparently unrelated, study, they were asked to recall events from their lives in which they had been either introverted or extroverted. Each group of subjects provided more memories connecting themselves with the more desirable personality type, and recalled those memories more quickly.
One study showed how selective memory can maintain belief in extrasensory perception (ESP). Believers and disbelievers were each shown descriptions of ESP experiments. Half of each group were told that the experimental results supported the existence of ESP, while the others were told they did not. In a subsequent test, subjects recalled the material accurately, apart from believers who had read the non-supportive evidence. This group remembered significantly less information and some of them incorrectly remembered the results as supporting ESP.
A similar cognitive bias found in individuals is the backfire effect, in which individuals challenged with evidence contradictory to their beliefs tend to reject the evidence and instead become an even firmer supporter of their initial belief. The phrase was first coined by Brendan Nyhan and Jason Reifler in a paper entitled "When Corrections Fail: The persistence of political misperceptions".
Backfire effectA similar cognitive bias found in individuals is the backfire effect, in which individuals challenged with evidence contradictory to their beliefs tend to reject the evidence and instead become an even firmer supporter of their initial belief. The phrase was first coined by Brendan Nyhan and Jason Reifler in a paper entitled "When Corrections Fail: The persistence of political misperceptions".
Polarization of opinionWhen people with opposing views interpret new information in a biased way, their views can move even further apart. This is called "attitude polarization". The effect was demonstrated by an experiment that involved drawing a series of red and black balls from one of two concealed "bingo baskets". Subjects knew that one basket contained 60% black and 40% red balls; the other, 40% black and 60% red. The experimenters looked at what happened when balls of alternating color were drawn in turn, a sequence that does not favor either basket. After each ball was drawn, subjects in one group were asked to state out loud their judgments of the probability that the balls were being drawn from one or the other basket. These subjects tended to grow more confident with each successive draw—whether they initially thought the basket with 60% black balls or the one with 60% red balls was the more likely source, their estimate of the probability increased. Another group of subjects were asked to state probability estimates only at the end of a sequence of drawn balls, rather than after each ball. They did not show the polarization effect, suggesting that it does not necessarily occur when people simply hold opposing positions, but rather when they openly commit to them.
A less abstract study was the Stanford biased interpretation experiment in which subjects with strong opinions about the death penalty read about mixed experimental evidence. Twenty-three percent of the subjects reported that their views had become more extreme, and this self-reported shift correlated strongly with their initial attitudes. In later experiments, subjects also reported their opinions becoming more extreme in response to ambiguous information. However, comparisons of their attitudes before and after the new evidence showed no significant change, suggesting that the self-reported changes might not be real. Based on these experiments, Deanna Kuhn and Joseph Lao concluded that polarization is a real phenomenon but far from inevitable, only happening in a small minority of cases. They found that it was prompted not only by considering mixed evidence, but by merely thinking about the topic.
Charles Taber and Milton Lodge argued that the Stanford team's result had been hard to replicate because the arguments used in later experiments were too abstract or confusing to evoke an emotional response. The Taber and Lodge study used the emotionally charged topics of gun control and affirmative action. They measured the attitudes of their subjects towards these issues before and after reading arguments on each side of the debate. Two groups of subjects showed attitude polarization; those with strong prior opinions and those who were politically knowledgeable. In part of this study, subjects chose which information sources to read, from a list prepared by the experimenters. For example they could read the National Rifle Association's and the Brady Anti-Handgun Coalition's arguments on gun control. Even when instructed to be even-handed, subjects were more likely to read arguments that supported their existing attitudes. This biased search for information correlated well with the polarization effect.
Persistence of discredited beliefsConfirmation biases can be used to explain why some beliefs remain when the initial evidence for them is removed. This belief perseverance effect has been shown by a series of experiments using what is called the "debriefing paradigm": subjects read fake evidence for a hypothesis, their attitude change is measured, then the fakery is exposed in detail. Their attitudes are then measured once more to see if their belief returns to its previous level.
A typical finding is that at least some of the initial belief remains even after a full debrief. In one experiment, subjects had to distinguish between real and fake suicide notes. The feedback was random: some were told they had done well while others were told they had performed badly. Even after being fully debriefed, subjects were still influenced by the feedback. They still thought they were better or worse than average at that kind of task, depending on what they had initially been told.
In another study, subjects read job performance ratings of two firefighters, along with their responses to a risk aversion test. These fictional data were arranged to show either a negative or positive association: some subjects were told that a risk-taking firefighter did better, while others were told they did less well than a risk-averse colleague. Even if these two case studies had been true, they would have been scientifically poor evidence for a conclusion about firefighters in general. However, the subjects found them subjectively persuasive. When the case studies were shown to be fictional, subjects' belief in a link diminished, but around half of the original effect remained. Follow-up interviews established that the subjects had understood the debriefing and taken it seriously. Subjects seemed to trust the debriefing, but regarded the discredited information as irrelevant to their personal belief.
Preference for early informationExperiments have shown that information is weighted more strongly when it appears early in a series, even when the order is unimportant. For example, people form a more positive impression of someone described as "intelligent, industrious, impulsive, critical, stubborn, envious" than when they are given the same words in reverse order. This irrational primacy effect is independent of the primacy effect in memory in which the earlier items in a series leave a stronger memory trace. Biased interpretation offers an explanation for this effect: seeing the initial evidence, people form a working hypothesis that affects how they interpret the rest of the information.
One demonstration of irrational primacy involved colored chips supposedly drawn from two urns. Subjects were told the color distributions of the urns, and had to estimate the probability of a chip being drawn from one of them. In fact, the colors appeared in a pre-arranged order. The first thirty draws favored one urn and the next thirty favored the other. The series as a whole was neutral, so rationally, the two urns were equally likely. However, after sixty draws, subjects favored the urn suggested by the initial thirty.
Another experiment involved a slide show of a single object, seen as just a blur at first and in slightly better focus with each succeeding slide. After each slide, subjects had to state their best guess of what the object was. Subjects whose early guesses were wrong persisted with those guesses, even when the picture was sufficiently in focus that other people could readily identify the object.
Illusory association between eventsIllusory correlation is the tendency to see non-existent correlations in a set of data. This tendency was first demonstrated in a series of experiments in the late 1960s. In one experiment, subjects read a set of psychiatric case studies, including responses to the Rorschach inkblot test. They reported that the homosexual men in the set were more likely to report seeing buttocks, anuses or sexually ambiguous figures in the inkblots. In fact the case studies were fictional and, in one version of the experiment, had been constructed so that the homosexual men were less likely to report this imagery. In a survey, a group of experienced psychoanalysts reported the same set of illusory associations with homosexuality.
Another study recorded the symptoms experienced by arthritic patients, along with weather conditions over a 15-month period. Nearly all the patients reported that their pains were correlated with weather conditions, although the real correlation was zero.
This effect is a kind of biased interpretation, in that objectively neutral or unfavorable evidence is interpreted to support existing beliefs. It is also related to biases in hypothesis-testing behavior. In judging whether two events, such as illness and bad weather, are correlated, people rely heavily on the number of positive-positive cases: in this example, instances of both pain and bad weather. They pay relatively little attention to the other kinds of observation (of no pain and/or good weather). This parallels the reliance on positive tests in hypothesis testing. It may also reflect selective recall, in that people may have a sense that two events are correlated because it is easier to recall times when they happened together.
In the above fictional example, arthritic symptoms are more likely on days with no rain. However, people are likely to focus on the relatively large number of days which have both rain and symptoms. By concentrating on one cell of the table rather than all four, people can misperceive the relationship, in this case associating rain with arthritic symptoms.
Informal observationBefore psychological research on confirmation bias, the phenomenon had been observed anecdotally by writers, including the Greek historian Thucydides (c. 460 BC – c. 395 BC), Italian poet Dante Alighieri (1265–1321), English philosopher and scientist Francis Bacon (1561–1626), and Russian author Leo Tolstoy (1828–1910). Thucydides, in The Peloponnesian War wrote: "…for it is a habit of mankind to entrust to careless hope what they long for, and to use sovereign reason to thrust aside what they do not fancy." In the Divine Comedy, St. Thomas Aquinas cautions Dante when they meet in Paradise, "opinion—hasty—often can incline to the wrong side, and then affection for one's own opinion binds, confines the mind." Bacon, in the Novum Organum, wrote,
Bacon said that biased assessment of evidence drove "all superstitions, whether in astrology, dreams, omens, divine judgments or the like". In his essay "What Is Art?", Tolstoy wrote,
Wason's research on hypothesis-testingThe term "confirmation bias" was coined by English psychologist Peter Wason. For an experiment published in 1960, he challenged subjects to identify a rule applying to triples of numbers. At the outset, they were told that (2,4,6) fits the rule. Subjects could generate their own triples and the experimenter told them whether or not each triple conformed to the rule.
While the actual rule was simply "any ascending sequence", the subjects had a great deal of difficulty in finding it, often announcing rules that were far more specific, such as "the middle number is the average of the first and last". The subjects seemed to test only positive examples—triples that obeyed their hypothesized rule. For example, if they thought the rule was, "Each number is two greater than its predecessor", they would offer a triple that fit this rule, such as (11,13,15) rather than a triple that violates it, such as (11,12,19).
Wason accepted falsificationism, according to which a scientific test of a hypothesis is a serious attempt to falsify it. He interpreted his results as showing a preference for confirmation over falsification, hence the term "confirmation bias".[Note 3] Wason also used confirmation bias to explain the results of his selection task experiment. In this task, subjects are given partial information about a set of objects, and have to specify what further information they would need to tell whether or not a conditional rule ("If A, then B") applies. It has been found repeatedly that people perform badly on various forms of this test, in most cases ignoring information that could potentially refute the rule.
Klayman and Ha's critiqueA 1987 paper by Joshua Klayman and Young-Won Ha argued that the Wason experiments had not actually demonstrated a bias towards confirmation. Instead, Klayman and Ha interpreted the results in terms of a tendency to make tests that are consistent with the working hypothesis. They called this the "positive test strategy". This strategy is an example of a heuristic: a reasoning shortcut that is imperfect but easy to compute. Klayman and Ha used Bayesian probability and information theory as their standard of hypothesis-testing, rather than the falsificationism used by Wason. According to these ideas, each answer to a question yields a different amount of information, which depends on the person's prior beliefs. Thus a scientific test of a hypothesis is one that is expected to produce the most information. Since the information content depends on initial probabilities, a positive test can either be highly informative or uninformative. Klayman and Ha argued that when people think about realistic problems, they are looking for a specific answer with a small initial probability. In this case, positive tests are usually more informative than negative tests. However, in Wason's rule discovery task the answer—three numbers in ascending order—is very broad, so positive tests are unlikely to yield informative answers. Klayman and Ha supported their analysis by citing an experiment that used the labels "DAX" and "MED" in place of "fits the rule" and "doesn't fit the rule". This avoided implying that the aim was to find a low-probability rule. Subjects had much more success with this version of the experiment.
In light of this and other critiques, the focus of research moved away from confirmation versus falsification to examine whether people test hypotheses in an informative way, or an uninformative but positive way. The search for "true" confirmation bias led psychologists to look at a wider range of effects in how people process information.
Confirmation bias is often described as a result of automatic, unintentional strategies rather than deliberate deception. According to Robert Maccoun, most biased evidence processing occurs through a combination of both "cold" (cognitive) and "hot" (motivated) mechanisms.
Cognitive explanations for confirmation bias are based on limitations in people's ability to handle complex tasks, and the shortcuts, called heuristics, that they use. For example, people may judge the reliability of evidence by using the availability heuristic, i.e. how readily a particular idea comes to mind. It is also possible that people can only focus on one thought at a time, so find it difficult to test alternative hypotheses in parallel. Another heuristic is the positive test strategy identified by Klayman and Ha, in which people test a hypothesis by examining cases where they expect a property or event to occur. This heuristic avoids the difficult or impossible task of working out how diagnostic each possible question will be. However, it is not universally reliable, so people can overlook challenges to their existing beliefs.
Motivational explanations involve an effect of desire on belief, sometimes called "wishful thinking". It is known that people prefer pleasant thoughts over unpleasant ones in a number of ways: this is called the "Pollyanna principle". Applied to arguments or sources of evidence, this could explain why desired conclusions are more likely to be believed true. According to experiments that manipulate the desirability of the conclusion, people demand a high standard of evidence for unpalatable ideas and a low standard for preferred ideas. In other words, they ask, "Can I believe this?" for some suggestions and, "Must I believe this?" for others. Although consistency is a desirable feature of attitudes, an excessive drive for consistency is another potential source of bias because it may prevent people from neutrally evaluating new, surprising information. Social psychologist Ziva Kunda combines the cognitive and motivational theories, arguing that motivation creates the bias, but cognitive factors determine the size of the effect.
Explanations in terms of cost-benefit analysis assume that people do not just test hypotheses in a disinterested way, but assess the costs of different errors. Using ideas from evolutionary psychology, James Friedrich suggests that people do not primarily aim at truth in testing hypotheses, but try to avoid the most costly errors. For example, employers might ask one-sided questions in job interviews because they are focused on weeding out unsuitable candidates. Yaacov Trope and Akiva Liberman's refinement of this theory assumes that people compare the two different kinds of error: accepting a false hypothesis or rejecting a true hypothesis. For instance, someone who underestimates a friend's honesty might treat him or her suspiciously and so undermine the friendship. Overestimating the friend's honesty may also be costly, but less so. In this case, it would be rational to seek, evaluate or remember evidence of their honesty in a biased way. When someone gives an initial impression of being introverted or extroverted, questions that match that impression come across as more empathic. This suggests that when talking to someone who seems to be an introvert, it is a sign of better social skills to ask, "Do you feel awkward in social situations?" rather than, "Do you like noisy parties?" The connection between confirmation bias and social skills was corroborated by a study of how college students get to know other people. Highly self-monitoring students, who are more sensitive to their environment and to social norms, asked more matching questions when interviewing a high-status staff member than when getting to know fellow students.
Psychologists Jennifer Lerner and Philip Tetlock distinguish two different kinds of thinking process. Exploratory thought neutrally considers multiple points of view and tries to anticipate all possible objections to a particular position, while confirmatory thought seeks to justify a specific point of view. Lerner and Tetlock say that when people expect to need to justify their position to other people, whose views they already know, they will tend to adopt a similar position to those people, and then use confirmatory thought to bolster their own credibility. However, if the external parties are overly aggressive or critical, people will disengage from thought altogether, and simply assert their personal opinions without justification. Lerner and Tetlock say that people only push themselves to think critically and logically when they know in advance they will need to explain themselves to others who are well-informed, genuinely interested in the truth, and whose views they don't already know. Because those conditions rarely exist, they argue, most people are using confirmatory thought most of the time.
In financeConfirmation bias can lead investors to be overconfident, ignoring evidence that their strategies will lose money. In studies of political stock markets, investors made more profit when they resisted bias. For example, participants who interpreted a candidate's debate performance in a neutral rather than partisan way were more likely to profit. To combat the effect of confirmation bias, investors can try to adopt a contrary viewpoint "for the sake of argument". In one technique, they imagine that their investments have collapsed and ask themselves why this might happen.
In physical and mental healthRaymond Nickerson, a psychologist, blames confirmation bias for the ineffective medical procedures that were used for centuries before the arrival of scientific medicine. If a patient recovered, medical authorities counted the treatment as successful, rather than looking for alternative explanations such as that the disease had run its natural course. Biased assimilation is a factor in the modern appeal of alternative medicine, whose proponents are swayed by positive anecdotal evidence but treat scientific evidence hyper-critically.
Cognitive therapy was developed by Aaron T. Beck in the early 1960s and has become a popular approach. According to Beck, biased information processing is a factor in depression. His approach teaches people to treat evidence impartially, rather than selectively reinforcing negative outlooks. Phobias and hypochondria have also been shown to involve confirmation bias for threatening information.
In politics and lawNickerson argues that reasoning in judicial and political contexts is sometimes subconsciously biased, favoring conclusions that judges, juries or governments have already committed to. Since the evidence in a jury trial can be complex, and jurors often reach decisions about the verdict early on, it is reasonable to expect an attitude polarization effect. The prediction that jurors will become more extreme in their views as they see more evidence has been borne out in experiments with mock trials. Both inquisitorial and adversarial criminal justice systems are affected by confirmation bias.
Confirmation bias can be a factor in creating or extending conflicts, from emotionally charged debates to wars: by interpreting the evidence in their favor, each opposing party can become overconfident that it is in the stronger position. On the other hand, confirmation bias can result in people ignoring or misinterpreting the signs of an imminent or incipient conflict. For example, psychologists Stuart Sutherland and Thomas Kida have each argued that US Admiral Husband E. Kimmel showed confirmation bias when playing down the first signs of the Japanese attack on Pearl Harbor.
A two-decade study of political pundits by Philip E. Tetlock found that, on the whole, their predictions were not much better than chance. Tetlock divided experts into "foxes" who maintained multiple hypotheses, and "hedgehogs" who were more dogmatic. In general, the hedgehogs were much less accurate. Tetlock blamed their failure on confirmation bias—specifically, their inability to make use of new information that contradicted their existing theories.
In the paranormalOne factor in the appeal of psychic "readings" is that listeners apply a confirmation bias which fits the psychic's statements to their own lives. By making a large number of ambiguous statements in each sitting, the psychic gives the client more opportunities to find a match. This is one of the techniques of cold reading, with which a psychic can deliver a subjectively impressive reading without any prior information about the client. Investigator James Randi compared the transcript of a reading to the client's report of what the psychic had said, and found that the client showed a strong selective recall of the "hits".
As a striking illustration of confirmation bias in the real world, Nickerson mentions numerological pyramidology: the practice of finding meaning in the proportions of the Egyptian pyramids. There are many different length measurements that can be made of, for example, the Great Pyramid of Giza and many ways to combine or manipulate them. Hence it is almost inevitable that people who look at these numbers selectively will find superficially impressive correspondences, for example with the dimensions of the Earth.
In scienceA distinguishing feature of scientific thinking is the search for falsifying as well as confirming evidence. However, many times in the history of science, scientists have resisted new discoveries by selectively interpreting or ignoring unfavorable data. Previous research has shown that the assessment of the quality of scientific studies seems to be particularly vulnerable to confirmation bias. It has been found several times that scientists rate studies that report findings consistent with their prior beliefs more favorably than studies reporting findings inconsistent with their previous beliefs. However, assuming that the research question is relevant, the experimental design adequate and the data are clearly and comprehensively described, the found results should be of importance to the scientific community and should not be viewed prejudicially, regardless of whether they conform to current theoretical predictions.
Confirmation bias may thus be especially harmful to objective evaluations regarding nonconforming results since biased individuals may regard opposing evidence to be weak in principle and give little serious thought to revising their beliefs. Scientific innovators often meet with resistance from the scientific community, and research presenting controversial results frequently receives harsh peer review.
In the context of scientific research, confirmation biases can sustain theories or research programs in the face of inadequate or even contradictory evidence; the field of parapsychology has been particularly affected.
An experimenter's confirmation bias can potentially affect which data are reported. Data that conflict with the experimenter's expectations may be more readily discarded as unreliable, producing the so-called file drawer effect. To combat this tendency, scientific training teaches ways to prevent bias. For example, experimental design of randomized controlled trials (coupled with their systematic review) aims to minimize sources of bias. The social process of peer review is thought to mitigate the effect of individual scientists' biases, even though the peer review process itself may be susceptible to such biases.
In self-imageSocial psychologists have identified two tendencies in the way people seek or interpret information about themselves. Self-verification is the drive to reinforce the existing self-image and self-enhancement is the drive to seek positive feedback. Both are served by confirmation biases. In experiments where people are given feedback that conflicts with their self-image, they are less likely to attend to it or remember it than when given self-verifying feedback. They reduce the impact of such information by interpreting it as unreliable. Similar experiments have found a preference for positive feedback, and the people who give it, over negative feedback.
Note 1. David Perkins, a geneticist, coined the term "myside bias" referring to a preference for "my" side of an issue. (Baron 2000, p. 195)
Note 2. "Assimilation bias" is another term used for biased interpretation of evidence. (Risen & Gilovich 2007, p. 113)
Note 3. Wason also used the term "verification bias". (Poletiek 2001, p. 73)
1. a b Plous 1993, p. 233
2. Darley, John M.; Gross, Paget H. (2000), "A Hypothesis-Confirming Bias in Labelling Effects", in Stangor, Charles, Stereotypes and prejudice: essential readings, Psychology Press, p. 212, ISBN 978-0-86377-589-5, OCLC 42823720
3. Risen & Gilovich 2007
4. a b c Zweig, Jason (November 19, 2009), "How to Ignore the Yes-Man in Your Head", Wall Street Journal (Dow Jones & Company), retrieved 2010-06-13
5. Nickerson 1998, pp. 177–178
6. a b c d Kunda 1999, pp. 112–115
7. a b Baron 2000, pp. 162–164
8. Kida 2006, pp. 162–165
9. Devine, Patricia G.; Hirt, Edward R.; Gehrke, Elizabeth M. (1990), "Diagnostic and confirmation strategies in trait hypothesis testing", Journal of Personality and Social Psychology (American Psychological Association) 58 (6): 952–963, doi:10.1037/0022-35184.108.40.2062, ISSN 1939-1315
10. Trope, Yaacov; Bassok, Miriam (1982), "Confirmatory and diagnosing strategies in social information gathering", Journal of Personality and Social Psychology (American Psychological Association) 43 (1): 22–34, doi:10.1037/0022-35220.127.116.11, ISSN 1939-1315
11. a b c Klayman, Joshua; Ha, Young-Won (1987), "Confirmation, Disconfirmation and Information in Hypothesis Testing", Psychological Review (American Psychological Association) 94 (2): 211–228, doi:10.1037/0033-295X.94.2.211, ISSN 0033-295X, retrieved 2009-08-14
12. a b c Oswald & Grosjean 2004, pp. 82–83
13. Kunda, Ziva; Fong, G.T.; Sanitoso, R.; Reber, E. (1993), "Directional questions direct self-conceptions", Journal of Experimental Social Psychology (Society of Experimental Social Psychology) 29: 62–63, ISSN 0022-1031 via Fine 2006, pp. 63–65
14. a b Shafir, E. (1993), "Choosing versus rejecting: why some options are both better and worse than others", Memory and Cognition 21 (4): 546–556, PMID 8350746 via Fine 2006, pp. 63–65
15. Snyder, Mark; Swann, Jr., William B. (1978), "Hypothesis-Testing Processes in Social Interaction", Journal of Personality and Social Psychology (American Psychological Association) 36 (11): 1202–1212, doi:10.1037/0022-3518.104.22.1682 via Poletiek 2001, p. 131
16. a b Kunda 1999, pp. 117–118
17. a b Mynatt, Clifford R.; Doherty, Michael E.; Tweney, Ryan D. (1978), "Consequences of confirmation and disconfirmation in a simulated research environment", Quarterly Journal of Experimental Psychology 30 (3): 395–406, doi:10.1080/00335557843000007
18. Kida 2006, p. 157
19. a b c d e f Lord, Charles G.; Ross, Lee; Lepper, Mark R. (1979), "Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence", Journal of Personality and Social Psychology (American Psychological Association) 37 (11): 2098–2109, doi:10.1037/0022-3522.214.171.1248, ISSN 0022-3514
20. a b Baron 2000, pp. 201–202
21. Vyse 1997, p. 122
22. a b c d Taber, Charles S.; Lodge, Milton (July 2006), "Motivated Skepticism in the Evaluation of Political Beliefs", American Journal of Political Science (Midwest Political Science Association) 50 (3): 755–769, doi:10.1111/j.1540-5907.2006.00214.x, ISSN 0092-5853
23. a b Westen, Drew; Blagov, Pavel S.; Harenski, Keith; Kilts, Clint; Hamann, Stephan (2006), "Neural Bases of Motivated Reasoning: An fMRI Study of Emotional Constraints on Partisan Political Judgment in the 2004 U.S. Presidential Election", Journal of Cognitive Neuroscience (Massachusetts Institute of Technology) 18 (11): 1947–1958, doi:10.1162/jocn.2006.18.11.1947, PMID 17069484, retrieved 2009-08-14
24. Gadenne, V.; Oswald, M. (1986), "Entstehung und Veränderung von Bestätigungstendenzen beim Testen von Hypothesen [Formation and alteration of confirmatory tendencies during the testing of hypotheses]", Zeitschrift für experimentelle und angewandte Psychologie 33: 360–374 via Oswald & Grosjean 2004, p. 89
25. Hastie, Reid; Park, Bernadette (2005), "The Relationship Between Memory and Judgment Depends on Whether the Judgment Task is Memory-Based or On-Line", in Hamilton, David L., Social cognition: key readings, New York: Psychology Press, p. 394, ISBN 0-86377-591-8, OCLC 55078722
26. a b c Oswald & Grosjean 2004, pp. 88–89
27. Stangor, Charles; McMillan, David (1992), "Memory for expectancy-congruent and expectancy-incongruent information: A review of the social and social developmental literatures", Psychological Bulletin (American Psychological Association) 111 (1): 42–61, doi:10.1037/0033-2909.111.1.42
28. a b Snyder, M.; Cantor, N. (1979), "Testing hypotheses about other people: the use of historical knowledge", Journal of Experimental Social Psychology 15 (4): 330–342, doi:10.1016/0022-1031(79)90042-8 via Goldacre 2008, p. 231
29. Kunda 1999, pp. 225–232
30. Sanitioso, Rasyid; Kunda, Ziva; Fong, G.T. (1990), "Motivated recruitment of autobiographical memories", Journal of Personality and Social Psychology (American Psychological Association) 59 (2): 229–241, doi:10.1037/0022-35126.96.36.199, ISSN 0022-3514, PMID 2213492
31. a b Russell, Dan; Jones, Warren H. (1980), "When superstition fails: Reactions to disconfirmation of paranormal beliefs", Personality and Social Psychology Bulletin (Society for Personality and Social Psychology) 6 (1): 83–88, doi:10.1177/014616728061012, ISSN 1552-7433 via Vyse 1997, p. 121
32. "backfire effect". The Skeptic's Dictionary. Retrieved 26 April 2012.
33. Silverman, Craig (2011-06-17). "The Backfire Effect". Columbia Journalism Review. Retrieved 2012-05-01. "When your deepest convictions are challenged by contradictory evidence, your beliefs get stronger."
34. Nyhan, Brendan; Reifler, Jason (2010). "When Corrections Fail: The Persistence of Political Misperceptions". Political Behavior 32 (2): 303–330. doi:10.1007/s11109-010-9112-2. Retrieved 1 May 2012.
35. a b c Kuhn, Deanna; Lao, Joseph (March 1996), "Effects of Evidence on Attitudes: Is Polarization the Norm?", Psychological Science (American Psychological Society) 7 (2): 115–120, doi:10.1111/j.1467-9280.1996.tb00340.x
36. Baron 2000, p. 201
37. Miller, A.G.; McHoskey, J.W.; Bane, C.M.; Dowd, T.G. (1993), "The attitude polarization phenomenon: Role of response measure, attitude extremity, and behavioral consequences of reported attitude change", Journal of Personality and Social Psychology 64 (4): 561–574, doi:10.1037/0022-35188.8.131.521
38. a b c d Ross, Lee; Anderson, Craig A. (1982), "Shortcomings in the attribution process: On the origins and maintenance of erroneous social assessments", in Kahneman, Daniel; Slovic, Paul; Tversky, Amos, Judgment under uncertainty: Heuristics and biases, Cambridge University Press, pp. 129–152, ISBN 978-0-521-28414-1, OCLC 7578020
39. a b c d Nickerson 1998, p. 187
40. Kunda 1999, p. 99
41. Ross, Lee; Lepper, Mark R.; Hubbard, Michael (1975), "Perseverance in self-perception and social perception: Biased attributional processes in the debriefing paradigm", Journal of Personality and Social Psychology (American Psychological Association) 32 (5): 880–892, doi:10.1037/0022-35184.108.40.2060, ISSN 0022-3514, PMID 1185517 via Kunda 1999, p. 99
42. a b c Anderson, Craig A.; Lepper, Mark R.; Ross, Lee (1980), "Perseverance of Social Theories: The Role of Explanation in the Persistence of Discredited Information", Journal of Personality and Social Psychology (American Psychological Association) 39 (6): 1037–1049, doi:10.1037/h0077720, ISSN 0022-3514
43. a b c d e Baron 2000, pp. 197–200
44. a b c Fine 2006, pp. 66–70
45. a b Plous 1993, pp. 164–166
46. Redelmeir, D. A.; Tversky, Amos (1996), "On the belief that arthritis pain is related to the weather", Proceedings of the National Academy of Sciences 93 (7): 2895–2896, doi:10.1073/pnas.93.7.2895 via Kunda 1999, p. 127
47. a b c Kunda 1999, pp. 127–130
48. Plous 1993, pp. 162–164
49. Adapted from Fielder, Klaus (2004), "Illusory correlation", in Pohl, Rüdiger F., Cognitive Illusions: A Handbook on Fallacies and Biases in Thinking, Judgement and Memory, Hove, UK: Psychology Press, p. 103, ISBN 978-1-84169-351-4, OCLC 55124398
50. a b Baron 2000, pp. 195–196
51. Thucydides 4.108.4
52. Alighieri, Dante. Paradiso canto XIII: 118–120. Trans. Allen Mandelbaum
53. a b Bacon, Francis (1620). Novum Organum. reprinted in Burtt, E.A., ed. (1939), The English philosophers from Bacon to Mill, New York: Random House, p. 36 via Nickerson 1998, p. 176
54. Tolstoy, Leo. What is Art? p. 124 (1899). In The Kingdom of God Is Within You (1893), he similarly declared, "The most difficult subjects can be explained to the most slow-witted man if he has not formed any idea of them already; but the simplest thing cannot be made clear to the most intelligent man if he is firmly persuaded that he knows already, without a shadow of doubt, what is laid before him" (ch. 3). Translated from the Russian by Constance Garnett, New York, 1894. Project Gutenberg edition released November 2002. Retrieved 2009-08-24.
55. Gale, Maggie; Ball, Linden J. (2002), "Does Positivity Bias Explain Patterns of Performance on Wason's 2-4-6 task?", in Gray, Wayne D.; Schunn, Christian D., Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science Society, Routledge, p. 340, ISBN 978-0-8058-4581-5, OCLC 469971634
56. a b Wason, Peter C. (1960), "On the failure to eliminate hypotheses in a conceptual task", Quarterly Journal of Experimental Psychology (Psychology Press) 12 (3): 129–140, doi:10.1080/17470216008416717, ISSN 1747-0226
57. Nickerson 1998, p. 179
58. Lewicka 1998, p. 238
59. Oswald & Grosjean 2004, pp. 79–96
60. Wason, Peter C. (1968), "Reasoning about a rule", Quarterly Journal of Experimental Psychology (Psychology Press) 20 (3): 273–28, doi:10.1080/14640746808400161, ISSN 1747-0226
61. a b c Sutherland, Stuart (2007), Irrationality (2nd ed.), London: Pinter and Martin, pp. 95–103, ISBN 978-1-905177-07-3, OCLC 72151566
62. Barkow, Jerome H.; Cosmides, Leda; Tooby, John (1995), The adapted mind: evolutionary psychology and the generation of culture, Oxford University Press US, pp. 181–184, ISBN 978-0-19-510107-2, OCLC 33832963
63. Oswald & Grosjean 2004, pp. 81–82, 86–87
64. Lewicka 1998, p. 239
65. Tweney, Ryan D.; Doherty, Michael E.; Worner, Winifred J.; Pliske, Daniel B.; Mynatt, Clifford R.; Gross, Kimberly A.; Arkkelin, Daniel L. (1980),"Strategies of rule discovery in an inference task", The Quarterly Journal of Experimental Psychology (Psychology Press) 32 (1): 109–123, doi:10.1080/00335558008248237, ISSN 1747-0226 (Experiment IV)
66. Oswald & Grosjean 2004, pp. 86–89
67. a b Hergovich, Schott & Burger 2010
68. Maccoun 1998
69. Friedrich 1993, p. 298
70. Kunda 1999, p. 94
71. Nickerson 1998, pp. 198–199
72. Nickerson 1998, p. 200
73. a b c Nickerson 1998, p. 197
74. Baron 2000, p. 206
75. Matlin, Margaret W. (2004), "Pollyanna Principle", in Pohl, Rüdiger F., Cognitive Illusions: A Handbook on Fallacies and Biases in Thinking, Judgement and Memory, Hove: Psychology Press, pp. 255–272, ISBN 978-1-84169-351-4, OCLC 55124398
76. Dawson, Erica; Gilovich, Thomas; Regan, Dennis T. (October 2002), "Motivated Reasoning and Performance on the Wason Selection Task", Personality and Social Psychology Bulletin (Society for Personality and Social Psychology) 28 (10): 1379–1387, doi:10.1177/014616702236869, retrieved 2009-09-30
77. Ditto, Peter H.; Lopez, David F. (1992), "Motivated skepticism: use of differential decision criteria for preferred and nonpreferred conclusions", Journal of personality and social psychology (American Psychological Association) 63 (4): 568–584, doi:10.1037/0022-35220.127.116.118, ISSN 0022-3514
78. Nickerson 1998, p. 198
79. Oswald & Grosjean 2004, pp. 91–93
80. Friedrich 1993, pp. 299, 316–317
81. Trope, Y.; Liberman, A. (1996), "Social hypothesis testing: cognitive and motivational mechanisms", in Higgins, E. Tory; Kruglanski, Arie W., Social Psychology: Handbook of basic principles, New York: Guilford Press, ISBN 978-1-57230-100-9, OCLC 34731629 via Oswald & Grosjean 2004, pp. 91–93
82. a b Dardenne, Benoit; Leyens, Jacques-Philippe (1995), "Confirmation Bias as a Social Skill", Personality and Social Psychology Bulletin (Society for Personality and Social Psychology) 21 (11): 1229–1239, doi:10.1177/01461672952111011, ISSN 1552-7433
83. Sandra L. Schneider, ed. (2003). Emerging perspectives on judgment and decision research. Cambridge (u.a.): Cambridge University Press. p. 445. ISBN 052152718X.
84. Haidt, Jonathan (2012). The Righteous Mind : Why Good People are Divided by Politics and Religion. New York: Pantheon Books. pp. 1473–4 (e–book edition). ISBN 0307377903.
85. Lindzey, edited by Susan T. Fiske, Daniel T. Gilbert, Gardner (2010). The handbook of social psychology. (5th ed.). Hoboken, N.J.: Wiley. p. 811. ISBN 0470137495.
86. Pompian, Michael M. (2006), Behavioral finance and wealth management: how to build optimal portfolios that account for investor biases, John Wiley and Sons, pp. 187–190, ISBN 978-0-471-74517-4, OCLC 61864118
87. Hilton, Denis J. (2001), "The psychology of financial decision-making: Applications to trading, dealing, and investment analysis", Journal of Behavioral Finance (Institute of Behavioral Finance) 2 (1): 37–39, doi:10.1207/S15327760JPFM0201_4, ISSN 1542-7579
88. Krueger, David; Mann, John David (2009), The Secret Language of Money: How to Make Smarter Financial Decisions and Live a Richer Life, McGraw Hill Professional, pp. 112–113, ISBN 978-0-07-162339-1, OCLC 277205993
89. a b Nickerson 1998, p. 192
90. Goldacre 2008, p. 233
91. Singh, Simon; Ernst, Edzard (2008), Trick or Treatment?: Alternative Medicine on Trial, London: Bantam, pp. 287–288, ISBN 978-0-593-06129-9
92. Atwood, Kimball (2004), "Naturopathy, Pseudoscience, and Medicine: Myths and Fallacies vs Truth", Medscape General Medicine 6 (1): 33
93. Neenan, Michael; Dryden, Windy (2004), Cognitive therapy: 100 key points and techniques, Psychology Press, p. ix, ISBN 978-1-58391-858-6, OCLC 474568621
94. Blackburn, Ivy-Marie; Davidson, Kate M. (1995), Cognitive therapy for depression & anxiety: a practitioner's guide (2 ed.), Wiley-Blackwell, p. 19, ISBN 978-0-632-03986-9, OCLC 32699443
95. Harvey, Allison G.; Watkins, Edward; Mansell, Warren (2004), Cognitive behavioural processes across psychological disorders: a transdiagnostic approach to research and treatment, Oxford University Press, pp. 172–173, 176, ISBN 978-0-19-852888-3, OCLC 602015097
96. Nickerson 1998, pp. 191–193
97. Myers, D.G.; Lamm, H. (1976), "The group polarization phenomenon", Psychological Bulletin 83 (4): 602–627, doi:10.1037/0033-2909.83.4.602 via Nickerson 1998, pp. 193–194
98. Halpern, Diane F. (1987), Critical thinking across the curriculum: a brief edition of thought and knowledge, Lawrence Erlbaum Associates, p. 194, ISBN 978-0-8058-2731-6, OCLC 37180929
99. Roach, Kent (2010), "Wrongful Convictions: Adversarial and Inquisitorial Themes", North Carolina Journal of International Law and Commercial Regulation 35, SSRN 1619124, "Both adversarial and inquisitorial systems seem subject to the dangers of tunnel vision or confirmation bias."
100. Baron 2000, pp. 191,195
101. Kida 2006, p. 155
102. Tetlock, Philip E. (2005), Expert Political Judgment: How Good Is It? How Can We Know?, Princeton, N.J.: Princeton University Press, pp. 125–128, ISBN 978-0-691-12302-8, OCLC 56825108
103. a b Smith, Jonathan C. (2009), Pseudoscience and Extraordinary Claims of the Paranormal: A Critical Thinker's Toolkit, John Wiley and Sons, pp. 149–151, ISBN 978-1-4051-8122-8, OCLC 319499491
104. Randi, James (1991), James Randi: psychic investigator, Boxtree, pp. 58–62, ISBN 978-1-85283-144-8, OCLC 26359284
105. a b Nickerson 1998, p. 190
106. a b Nickerson 1998, pp. 192–194
107. a b Koehler 1993
108. a b c Mahoney 1977
109. Horrobin 1990
110. Proctor, Robert W.; Capaldi, E. John (2006), Why science matters: understanding the methods of psychological research, Wiley-Blackwell, p. 68, ISBN 978-1-4051-3049-3, OCLC 318365881
111. Sternberg, Robert J. (2007), "Critical Thinking in Psychology: It really is critical", in Sternberg, Robert J.; Roediger III, Henry L.; Halpern, Diane F., Critical Thinking in Psychology, Cambridge University Press, p. 292, ISBN 0-521-60834-1, OCLC 69423179, "Some of the worst examples of confirmation bias are in research on parapsychology ... Arguably, there is a whole field here with no powerful confirming data at all. But people want to believe, and so they find ways to believe."
112. a b Shadish, William R. (2007), "Critical Thinking in Quasi-Experimentation", in Sternberg, Robert J.; Roediger III, Henry L.; Halpern, Diane F., Critical Thinking in Psychology, Cambridge University Press, p. 49, ISBN 978-0-521-60834-3
113. Jüni, P.; Altman, D. G.; Egger, M. (2001). "Systematic reviews in health care: Assessing the quality of controlled clinical trials". BMJ (Clinical research ed.) 323 (7303): 42–46. PMC 1120670. PMID 11440947.
114. Shermer, Michael (July 2006), "The Political Brain", Scientific American, ISSN 0036-8733, retrieved 2009-08-14 115. Emerson, G. B.; Warme, W. J.; Wolf, F. M.; Heckman, J. D.; Brand, R. A.; Leopold, S. S. (2010). "Testing for the Presence of Positive-Outcome Bias in Peer Review: A Randomized Controlled Trial". Archives of Internal Medicine 170 (21): 1934–1939. doi:10.1001/archinternmed.2010.406. PMID 21098355.
116. a b Swann, William B.; Pelham, Brett W.; Krull, Douglas S. (1989), "Agreeable Fancy or Disagreeable Truth? Reconciling Self-Enhancement and Self-Verification", Journal of Personality and Social Psychology (American Psychological Association) 57 (5): 782–791, doi:10.1037/0022-3518.104.22.1682, ISSN 0022–3514, PMID 2810025
117. a b Swann, William B.; Read, Stephen J. (1981), "Self-Verification Processes: How We Sustain Our Self-Conceptions", Journal of Experimental Social Psychology (Academic Press) 17 (4): 351–372, doi:10.1016/0022-1031(81)90043-3, ISSN 0022–1031
118. Story, Amber L. (1998), "Self-Esteem and Memory for Favorable and Unfavorable Personality Feedback", Personality and Social Psychology Bulletin (Society for Personality and Social Psychology) 24 (1): 51–64, doi:10.1177/0146167298241004, ISSN 1552-7433
119. White, Michael J.; Brockett, Daniel R.; Overstreet, Belinda G. (1993), "Confirmatory Bias in Evaluating Personality Test Information: Am I Really That Kind of Person?", Journal of Counseling Psychology (American Psychological Association) 40 (1): 120–126, doi:10.1037/0022-022.214.171.124, ISSN 0022-0167
120. Swann, William B.; Read, Stephen J. (1981), "Acquiring Self-Knowledge: The Search for Feedback That Fits", Journal of Personality and Social Psychology (American Psychological Association) 41 (6): 1119–1128, ISSN 0022–3514
121. Shrauger, J. Sidney; Lund, Adrian K. (1975), "Self-evaluation and reactions to evaluations from others", Journal of Personality (Duke University Press) 43 (1): 94–108, doi:10.1111/j.1467-6494.1975.tb00574, PMID 1142062