This post on decision hygiene is an extension of my summary of the book Noise by Kahneman, Sibony and Sunstein. I have split it out from the overall summary because of length, and I have split it out also from my post on how to reduce noise because I think these methods are more about improving the quality of decisions generally, rather than reducing noise. Using these methods will also tend to reduce noise as higher quality decisions will be more likely to approach the “true” value.
Buy Noise at: Amazon | Kobo (affiliate links)
Kahneman, Sibony and Sunstein use the term decision hygiene to compare it to washing your hands. It’s good practice but you don’t see what errors/infections you avoided by doing it. As they say, it is invaluable but thankless.
They point out that ex-post or ex-ante debiasing works in situations where the general direction of error is known and strong. (Ex-post debiasing means you make correct your judgment after you make it. Ex-ante debiasing means you correct your judgment in advance.) For example, we know that the planning fallacy means that people systematically underestimate the amount of time needed to complete a project. So you can correct for planning fallacy bias by building in a fixed buffer (say, +35%) for project estimates, and set the buffer percentage by looking at past results.
But in many situations you won’t know the direction or error in advance. There can be a bunch of different biases working in different directions and it is not clear how strong each one will be. You can still use decision hygiene in these cases.
Kahneman, Sibony and Sunstein set out the following methods:
- Structuring complex judgments
- Mediating assessments protocol
- Using a decision observer
- Taking the outside view
- Sequencing information
1. Structuring Complex Judgments
Decomposing a judgment into its component parts ensures that the inputs are independent of each other. Final judgment should be delayed until all relevant inputs have been collected.
Three principles:
- Decomposition. Break down the decision into components. It focuses the judges on relevant predictors.
- Independence. Collect information on each assessment independently (or as independently as possible).
- Delayed holistic judgment. Delay the overall judgment until information on each component has been collected and assessed independently. This method still allows room for people to exercise their judgment, but just delays it.
One reason why this method reduces noise is because agreement is often fairly high in judgments on single dimensions. For example, when assessing two candidates for a job, people often agree on which one has more leadership potential or which one has better technical skills. They may just differ on which factor is more important.
Examples:
- Essay marking
- Behavioural interviews
- Apgar scores, used to evaluate the health of newborn babies.
2. Mediating Assessments Protocol (MAP)
A method of decision-making designed by Kahneman, Sibony and Lovallo. A primary objective of MAP is to reduce noise, and it incorporates a lot of other decision hygiene strategies. MAP is intended for big, important decisions like whether to acquire a company. It treats different decision options like different candidates for a job.
Steps involved:
- Agree on the approach
- Decide in advance on a comprehensive list of relevant factors (the authors uses the term “assessments”). Ideally the factors should be independent in that a fact should preferably influence only one factor, to minimise redundancy. Sometimes important but unforeseen factors may arise. You can take them into account, but there should be a high threshold (apply the broken-leg principle).
- Discuss the factors separately. The estimate-talk-estimate (mini-Delphi method) may be helpful. Use an outside view where possible (see below).
- Reach a final judgment on the deal, after having discussed all the factors separately.
3. Use a decision observer
Using a decision observer works because of a bias blind spot, where people recognise biases in others more easily than in themselves. (See also the self-serving bias.)
Kahneman, Sibony and Sunstein suggest that decision observers need to be trained and have some tools, like a checklist of biases. While they have seen some encouraging results in informal, small-scale efforts, they hope that others will do more research into this approach.
4. Taking the outside view (looking at base rates)
A base rate is the probability of something being true if you had no other information. Wikipedia gives the example that, if 1% of the public were “medical professionals”, and 99% of the public were not “medical professionals”, then the base rate of medical professionals is simply 1%.
Neglecting base rates is a common mistake that tends to reduce the accuracy of predictions. Taking the outside view (i.e. taking into account base rates), and then adjusting your prediction based on the information you do have, will usually give a much better prediction.
Predictions will also tend to be less noisy if everyone is taking the outside view and looking at base rates. Because then the base rate will be the initial anchor, rather than everyone picking an arbitrary initial anchor.
5. Sequencing information
Sequencing information is useful because it limits the forming of premature intuitions, and limits confirmation bias.
In some important domains (e.g. fingerprinting), you may even get judges to document their judgments at each step. Judges should also document changes in judgments and the reasons for that. [I wonder if this may have the unintended consequence of discouraging people from changing their initial judgment. Because they may want to avoid the hassle of documenting, and because they may get more attached to the initial judgment if they wrote it down.]
A related aspect of sequencing is when you call on multiple people to make judgments, you should ensure that they are unaware of each other’s judgments.
Buy Noise at: Amazon | Kobo <– These are affiliate links, which means I may earn a small commission if you buy through these links. I’d be grateful if you considered supporting the site in this way! 🙂
You may also want to check out: