anecdotal evidence (testimonials)

Testimonials and anecdotes are used to support claims in many fields. Advertisers often rely on testimonials to persuade consumers of the effectiveness or value of their products or services. Others use anecdotes to drive home the horror of some alleged activity or the danger of widely-used electronic devices like cell phones. In the mid-90s, there were many people, some in law enforcement, claiming that Satanists were abducting and abusing children on a massive scale. The anecdotes involved vivid descriptions of horrible sexual abuse, even murder of innocent children. The anecdotes were quite convincing, especially when they were repeated on nationally televised programs with popular hosts like Geraldo Rivera. A four-year study in the early 1990s found the allegations of satanic ritual abuse to be without merit. Researchers investigated more than 12,000 accusations and surveyed more than 11,000 psychiatric, social service, and law enforcement personnel. The researchers could find no unequivocal evidence for a single case of satanic cult ritual abuse.


There have also been scares fueled by anecdotes regarding such disparate items as silicone breast implants, cell phones, and vaccinations. In the 1990s many women blamed their cancers and other diseases on breast implants. Talk show hosts like Oprah Winfrey and Jenny Jones presented groups of women who were suffering from cancer or some other serious disease and who had been diagnosed after they'd had breast implants. The stories tugged at the heartstrings and brought tears to many sensitive eyes, but the scientific evidence did not exist that there was a causal connection between the implants and any disease. That fact did not prevent lawyers from extorting $4.25 billion from implant manufacturers. Marcia Angell, former executive editor of the New England Journal of Medicine, brought the wrath of feminist hell upon herself in 1992 when she wrote an editorial challenging the Food and Drug Administration's decision to ban the manufacture of silicone breast implants. The scientific evidence wasn't there to justify the ban. She eventually wrote a book describing the fiasco: Science on Trial: The Clash of Medical Evidence and the Law in the Breast Implant Case. The scientific evidence is now in. The implants don't cause cancer or other diseases, and the FDA has lifted its ban. When the data were collected, they showed that women with silicone breast implants did not suffer cancer or any other disease at a significantly higher rate than women who had not had implants.

The public fear that cellphones might be causing brain tumors was first aroused not by scientists but by a talk show host. On January 23, 1993, Larry King's guest was David Reynard, who announced that he and his wife Susan had sued NEC and GTE on the grounds that the cellphone David gave Susan caused his wife's brain tumor. There was nothing but junk science to back up her claim, plus the fact that the tumor appeared near where she held the phone to her ear. She was diagnosed seven months after receiving her phone and died a few months after filing the suit. The suit was dismissed in 1995. A dozen similar lawsuits followed; all were dismissed (1,2,3). For those who think scientists and industries don't take anecdotes seriously, consider this: soon after Susan's lawsuit was dismissed the cellphone industry committed $25 million for safety studies. Many studies have been conducted over the past fifteen years and so far no evidence of a causal link between cellphones and brain cancer has been found.

The vaccination rate in many places has dropped significantly in many parts of the world. In my northern California university town, 40% of the kindergartners at the Davis Waldorf school are unvaccinated. In the Sacramento area, there has been a 34% increase in "personal-belief exemptions" from state-required vaccinations for kindergartners over the past four years. Statewide, the increase in waivers has been 37% over the same period. The greatest decline in vaccination rates has occurred among the wealthier and more educated segments of society ("Child vaccine rates fall," Sacramento Bee, 1/6/2013), not because of scientific evidence that vaccines are harmful but mainly because of fear caused in large part by anecdotes of children getting autism and other neurological disorders from vaccinations. Oprah Winfrey, for example, responded to a systematic letter campaign from parents of kids with autism to feature on her show actress Jenny McCarthy and others to share their stories about getting a vaccination and then being diagnosed with autism. This kind of post hoc reasoning is common among those who believe anecdotes are more trustworthy than scientific studies. Scientific studies have repeatedly found no causal connection between vaccines and autism or serious neurological disorders. The benefits to members of society from universal vaccination against communicable diseases such as measles, mumps, polio, and diphtheria far outweigh any potential harm that might happen to somebody somewhere under some circumstances.

The fear of vaccines has led to outbreaks of measles and deaths to infants from whooping cough, events that should not be happening in this day and age. In Japan, when the vaccination rate for pertussis (whooping cough) dropped 70% from 1974 to 1976, the number of cases of pertussis went from 393 to more than 13,000 and the number of deaths from pertussis went from 0 to 41.*

Testimonials and vivid anecdotes are unreliable for various reasons. Stories are prone to contamination by beliefs, later experiences, feedback, selective attention to details, and so on. Most stories get distorted in the telling and the retelling. Events get exaggerated. Time sequences get confused. Details get muddled. Memories are imperfect and selective; they are often filled in after the fact. People misinterpret their experiences and are biased and selective in what interpretations they include and exclude from consideration. Experiences are conditioned by biases, memories, and beliefs, so people's perceptions might not be accurate. Most people aren't expecting to be deceived, so they may not be aware of deceptions that others might engage in. Some people make up stories. Some stories are delusions. Sometimes events are inappropriately deemed improbable when they might not be that improbable after all. In short, anecdotes are inherently problematic and are usually impossible to test for accuracy.

Some fields rely almost exclusively on anecdotal evidence and testimonials, e.g., alternative medicine, the paranormal, the supernatural, and the pseudoscientific. Stories of personal experience with acupuncture, mediums, ghosts of relatives, or free energy machines have little scientific value. Sincere and vivid accounts of one’s encounter with an angel or the Virgin Mary, an alien, a Bigfoot, a child claiming to have lived before, purple auras around dying patients, a miraculous dowser, a levitating guru, or a psychic surgeon are of little value in establishing the reasonableness of believing in such matters. If others cannot experience the same thing under the same conditions, then there will be no way to verify the experience. If there is no way to test the claim made, then there will be no way to tell if the experience was interpreted correctly. If others can experience the same thing, then it is possible to make a test of the testimonial and determine whether the claim based on it is worthy of belief. As parapsychologist Charles Tart once said after reporting an anecdote of a possibly paranormal event: “Let’s take this into the laboratory, where we can know exactly what conditions were. We don’t have to hear a story told years later and hope that it was accurate.” Another parapsychologist, Dean Radin, also noted that anecdotes aren't good proof of the paranormal because memory “is much more fallible than most people think” and eyewitness testimony “is easily distorted” (Radin 1997: 32).

Testimonials are of little use to science because selective thinking and self-deception can't be controlled for or mitigated in randomly experienced events as they must be in scientific observations and experiments. Most psychics and dowsers, for example, do not even realize that they need to do controlled tests of their powers to rule out the possibility that they are deceiving themselves. They are satisfied that their experiences provide them with enough positive feedback to justify the belief in their paranormal abilities. It is common for psychics, dowsers, and their admirers to remember their apparent successes and ignore or underplay their failures. Controlled tests can also determine whether other factors such as cheating might be involved.

If testimonials are scientifically worthless, why are they so popular and why are they so convincing? There are several reasons. Testimonials are often vivid and detailed, making the coincidental seem meaningful and giving a causal interpretation more credibility than it deserves. They are often made by enthusiastic people who seem trustworthy and honest and who seem to lack any reason to deceive us. Sometimes a testimonial is given soon after an experience while one’s mood is still elevated from the desire for a positive outcome. The experience and the testimonial it elicits are given more significance than they deserve and are of little value in establishing the probability of the claims they are put forth to support. Testimonials are often made by people with a semblance of authority, such as those who wear a uniform or hold a Ph.D. or M.D. degree. Testimonials are often made by popular figures given a  bully pulpit on widely viewed television programs. To some extent, testimonials are believable because people want to believe them. Testimonials accompanied by claims of government or Big Pharma conspiracies to stifle a new cancer cure or free energy device are popular among a certain class of people.

Testimonials and anecdotes are used to support claims in many fields, including medical science. Giving due consideration to such testimonials is considered wise, not foolish. A physician will use the testimonies of his or her patients to draw conclusions about certain medications or procedures. For example, a physician will take anecdotal evidence from a patient about a reaction to a new medication and use that information in deciding to adjust the prescribed dosage or to change the medication. This is quite reasonable. But the physician cannot be selective in listening to testimony, listening only to those claims that fit his or her own prejudices. To do so is to risk harming one’s patients. Nor should the rest of us be selective when listening to testimonials regarding some experience.

sources

Carroll, Robert Todd. 2012. Unnatural Acts: Critical Thinking, Skepticism, and Science Exposed! James Randi Educational Foundation.

Giere, Ronald, 1998. Understanding Scientific Reasoning, 4th ed, Holt Rinehart, Winston.

Kahane, Howard. 1997. Logic and Contemporary Rhetoric: The Use of Reason in Everyday Life, 8th edition. Wadsworth.