The “Shock Test” from How To Decide by Annie Duke

Surprised Boy

I recently took the “shock test” in Annie Duke’s book, How to Decide. The test requires you to produce estimates, along with a lower-bound and upper-bound estimate, for 10 figures.

Buy How to Decide at: Amazon | Kobo (affiliate links)

Your goal is to get 9 of the 10 items to fall within your ranges, not to get all 10. If you’re trying to get all 10 items to fall within your ranges, you’ll be overly conservative and set very wide ranges. It’s called the “shock test” because you should aim to be “pretty shocked” when an answer falls outside of your ranges. You’ll get the most value out of the test if you think carefully about each item and try to make the best estimates you can.

The test

Here are the 10 items if you want to try it yourself:

  1. The current population of the town in which you were born.
  2. Meryl Streep’s number of Academy Award nominations.
  3. Prince’s age at his death.
  4. The first year Nobel Prizes were awarded.
  5. The number of teams in the National Football League.
  6. The probability that a person in the US lives in a city with a population above 1 million.
  7. The number of people who voted for Abraham Lincoln in the 1860 presidential election.
  8. The height of the tip of the Statue of Liberty.
  9. The number of Billboard No. 1 singles by the Beatles.
  10. The probability that the cause of death of an average adult in the United States will be heart disease.

Answers to the shock test

My results

Only a measly 3 of my 9 ranges included the correct answer, though I was very close in another 2 of my ranges. (I didn’t do the Meryl Streep question as I accidentally saw the answer beforehand). It was a real eye-opener for me, because I felt like I was already erring on the side of caution in setting my ranges. But the test showed I was still overconfident.

Duke explains, however, that most people who take the test are surprised by how many of their ranges don’t include the correct answer. Like me, most people score less than 50% – far short of the 90% target. This shows that we normally oversell our knowledge rather than undersell it.

Share the test with your friends and see how they do!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.