Anecdotes are not Evidence

Non-fiction books are notorious for relying on anecdotes instead of actual evidence to prove their points. Discerning readers must approach anecdotes with caution and be alert to their improper use.

What is an anecdote?

In broad terms, an anecdote is a story about a single person’s experience, recounted after the fact.

Anecdotes can be contrasted with research studies, which look at the experiences of a larger sample of people. The sample need not be very big — qualitative studies may rely on in-depth interviews and observations, which will limit the number of participants they can reasonably follow. For example, in Evicted, Matthew Desmond writes about the experiences of 8 different tenant families and their landlords over two years. Through his research, Desmond came to understood eviction and its impact on tenants in a way he could not have if he’d observed a larger number of tenants more superficially. While Desmond told stories about the lives of real people, I wouldn’t say that Desmond relied on “anecdotes” because he didn’t cherry-pick his data — he chose the families to follow at the outset, not knowing what would happen to them.

This is one of the crucial differences between an anecdote and a study: in a study, the sample is taken before the results are known. If someone takes a random sample of college dropouts and tracks what happens to all of them, you can call that a study. If someone looks around at examples of high-profile success stories and notices that some of them dropped out of college — that’s an anecdote.

Why are anecdotes so prevalent in non-fiction?

Non-fiction writers love anecdotes. In fact, it’s hard to find a bestseller that doesn’t use extensive anecdotes.

There are several reasons why I think anecdotes are so prevalent:

• Anecdotes are engaging. They tell a story, and we love stories. Stories are easy to follow and we can often identify with the people in them. An anecdote is often more memorable and has a higher chance of being spread than a study.
• Anecdotes are simple. Anecdotes tell a very simple story. This person did A, and it was met with a success. This person did B, and it failed. In contrast, results of studies are often complex. Maybe only 10% the people who did A succeeded, and there was no effect on the remaining participants. So does that mean we do A? Authors like to use anecdotes because they can simplify a complex point.
• Anecdotes are easy to find. You can find an anecdote to prove pretty much any point. Some chain smokers live long lives and some college dropouts end up founding multibillion dollar businesses. Heck, you can even make up an anecdote and most of the time, no one will be able to prove it.
• Personal anecdotes also create a connection with the reader. These tiny stories tell the reader something about the author and humanise them. Moreover, when the anecdote is about something that helped the author succeeds, it lends an air of authority to the author’s advice. After all, we all want to learn a successful person’s secrets, don’t we?

Why anecdotes are dangerous

Most authors do not use anecdotes intending to mislead their readers. Particularly when sharing a personal anecdote, they’ll usually do so with the firm belief that the anecdote proves their point. Experiments have shown that, when learning, humans tend to overweight our own experiences and underweight others, and authors are only human.

But the reasons why authors love anecdotes are also the same reasons why you should approach them with caution. They present a simple and compelling story, when the true story might be more complex, nuanced, and difficult to explain. Likewise, the fact that anecdotes are so easy to find is exactly why they are unreliable — because they are cherry-picked.

And it’s not just the fact that anecdotes are cherry-picked that makes them dangerous. It’s even worse than that. When narrating an anecdote, the author selects which details to highlight and which to omit. Depending on how it is told, you could point to Steve Jobs’ life as an example of why you should “follow your passion” or why you shouldn’t follow your passion but just “be so good that they can’t ignore you”. You could point to the fact that Serena and Venus Williams started playing tennis at the age of 4 to say that you need 10,000 hours of deliberate practice to truly excel at a sport. Or, you could point to the fact that they practised a range of different sports in their early years to argue that, actually, range is the key.

Appropriate use of anecdotes

Now, I don’t mean to suggest that anecdotes are always bad. Anecdotes certainly have a place in non-fiction as not everything can be proved by a study.

An author may reasonably say, “hey, this method worked for me, and it might work for you too”. I’m not suggesting that you interrogate the research behind every bit of advice before making life decisions. In cases where the costs of trying the author’s suggestion are relatively low, there’s little harm in just trying trying it. Moreover, by sharing what worked for them, the author may give researchers an idea of where to concentrate further study.

Anecdotes can also be effectively used in philosophy to illustrate a point. For example, Diogenes the Cynic was famous for not needing material possessions — he lived in a barrel and owned simply a cloak, a staff and a bowl. One day, he saw a child drinking from a stream with his cupped hands. Diogenes promptly jumped up and cast away his bowl, exclaiming that it was “superfluous baggage”. This anecdote illustrates that it is possible to live and be happy with very few possessions, but doesn’t prove that getting rid of your stuff will make you happy. Remember: causation is much trickier than that.

A responsible author should explicitly acknowledge the limitations of their anecdotes and ideally supplement them with research studies where available. A good example of this is David Epstein’s Range — although Epstein used many anecdotes throughout, he also referred to many scientific studies and explained that he wanted to provide some memorable stories about the benefits of range to offset the already-widespread anecdotes about the benefits of early specialisation.

Buy Range at: Amazon | Kobo (affiliate links)

Conclusion

So, to summarise:

• Anecdotes differ from actual studies in that they are cherry-picked after the fact, once the outcomes are already known.
• Authors love to use anecdotes because they are engaging, simple and easy to find. Personal anecdotes can also create a connection with the reader. Yet it is these very characteristics that can make anecdotes so dangerous.
• Anecdotes have a place in non-fiction if used responsibly. They can give you ideas for what might work, and those suggestions can be worth trying even if they are not thoroughly researched. Anecdotes can also illustrate a point, even if they can’t prove it.