Our friend that we will be discussing today is Philip E Tetlock. He sets up home at both Annenberg School and the Wharton School at the University of Pennsylvania. Annenberg is their Communications school and Wharton is their business school. So, this perfect for what he does. He intersects the line between how we communicate and how we do business. When I look at Tetlock, I see him as yet another voice in the wide world of how we make decisions.
I have taken and extraordinary interest in Behavioral Economics of late. I am convinced that most and many of life’s most difficult decisions can be solved if we would truly understand the concept of behavior economics and apply them to our lives.
Although, I have covered this subject before, the simple summary of the Behavioral Economics subject is that it discusses why we make illogical choices. Often times this is applied against buying behavior, only in our case, we are going to follow Tetlock looking at forecasting behavior. He is trying to figure out what makes a good forecaster. And why do we care?
I will hazard a prediction. When you are 80 years old, and in a quiet moment of reflection narrating for only yourself the most personal version of your life story, the telling that will be most compact and meaningful will be the series of choices you have made.
In the end, we are our choices. Build yourself a great story.
~Jeff Bezos, Amazon founder, Princeton University, 2010
I strongly believe that Mr. Bezos is brilliant and correct. We are all about our choices, but then we must ask ourselves. Many of our choices are built on the perception of what is going to happen in the future. The Bible touches on this subject also. Our Lord and Master defined two distinct pillars that we must pass through.
On the left, we pass through the pillar of “don’t worry.”
Therefore do not worry about tomorrow, for tomorrow will worry about itself. Each day has enough trouble of its own.
On the right we pass through the pillar of “look at the circumstance and predict.”
But He replied to them, "When it is evening, you say, 'It will be fair weather, for the sky is red.' "And in the morning, 'There will be a storm today, for the sky is red and threatening.' Do you know how to discern the appearance of the sky, but cannot discern the signs of the times?
So in in this light, our forecasts should not be something that drives us to lack of faith in God, but they should be something that allows us to see the path that is in front of us.
Tetlock is known for kicking off the “Good Judgement Project” that helps us understand that often times those that look like they really understand the future really don’t understand the future. He has taken this work, and created a book with Dan Gardner, a reporter called Superforecasting, where he lays out his findings.
In his research, he was able to find people that could predict the future, which he calls his “superforecasters.” These are people that look simply on the outside for facts, but they have behaviors and habits that causes them to be much better than the average person in determining what the future holds. And what they do is truly amazing.
The common way of describing somebody’s ability to predict the future is call their Brier score. Similar to a golf handicap, you want to have a low Brier score. If you call everything correct, you are going to have a Brier score of 0. If you call out that something is going to happen, and if always does the opposite, then you are going to have a Brier score of 1.
The first thing to think through is the difficultly of the environment. If you live in Southern California, as I do, you may be asked to forecast if the weather is going to be good. You can generally predict that the weather is going to be good because it is always good. Thus being a good weather forecaster in Southern California is going to be easy. However, if you are in a geography that has unstable weather pattern, you are going to have a tough time making the call. So, your Brier score is going to be lower in easy to predict environments.
The problem is that most people don’t consider the environment in which they forecast. They assume that if they were successful it is because they had the right thinking. However, choice, by its very nature, deals with uncertainty. An individual may make a bunch of choices, and just simply be lucky. It is only by time and history do we see if the individual was lucky or had a way of thinking that really helped them make the right choice.
After years of research, Tutlock’s superforecaster are much more accurate. A large part of this is due to something that is referred to as the Hedgehogs and the Foxes. This idea was called out by Isaiah Berlin, and is pretty simple. When you think about things, does it all come down to “one big thing” or is it really “a bunch of smaller things that don’t necessarily have a clear winner.
Now the idea that “life is really not all that hard” is a great thought. It also turns out that it is very wrong. When we look at the future, those that are thinking about it through many different lenses do much better than try and make it simple. But here is the rub, those that seem to be able to declare at the top of their voice, “I can simplify it down to one thing” make everybody happy, and the research shows that they are promoted. Those that are much more fox like in their decision making simply have a tendency to be perceived as not decisive, therefore not as bright.
[By the way, to go off the track for a while, let us decide that we are going to do the right thing to become more thoughtful, more analytical, and therefore more accurate in our thinking. While this is the right thing to do, it doesn’t allow people to follow us. They are going to be worried that we don’t know what is happening. So you have three options:
1. Educate your audience about the Foxes and the Hedgehogs, so they become enlightened
2. Don’t educate them, and simply realize that they won’t follow you.
3. Portray yourself as a hedgehog, but silently be a fox.
I’ll give you a guess on which path you should take, and remind you that your choice must be relevant to the culture you are in.]
These foxes tend to work in “Active open-mindedness (AOM).” What is this? An idea created by Jonathan Baron. To be AOM, you need to be willing to look at other peoples ideas, and willing to challenge your own ideas.
And Baron would ask you the following questions to to see if you qualify as a AOM:
a. People should take into consideration evidence that goes against their beliefs.
b. It is more useful to pay attention to those who disagree with you than to pay attention to those who agree.
c. Changing your mind is a sign of weakness.
d. Intuition is the best guide in making decisions.
e. It is important to persevere in your beliefs even when evidence is brought to bear against them.
The book is an exceptional read, and while I do not have the blog space to summarize all of his key insights, he does leave us with the top 10 take aways from superforecasters:
Here they are in my own words:
1. Don’t try and answer unknowable questions. Answer what you can.
2. Break big problems into smaller problems. Do the Fermi.
3. Get a base rate. See if it has happened before and use that pattern.
4. Don’t immediately react to new evidence, but don’t ignore it.
5. Argue both side of a forecast.
6. Make sure that you get all the areas of doubt defined, but don’t stop because you find them all.
7. Know that you can be overconfidence and underconfident. Take a position, but not too fast.
8. Keep a scorecard, and stop saying ‘yup, I saw it.” You are probably deluding yourself unless you wrote it down with a percentage. Once you do this, people get much better at the future forecast.
9. Find a group that fights with you, but yet gets along with you. Spirited team debate is good, as long as it is done with good intentions and nobody gets hurt.
10. Do. You can’t get better if you don’t start the above.
As a final note:
"Psychologists find that many atheists also see meaning in the significant events in their lives, and a majority of atheists said they believe in fate, defined as the view that “events happen for a reason and that there is an underlying order to life that determines how events turn out.” (Meaning is a basic human need)
By the way, this basically blew me away as a factoid, and yet was buried in this book. This came out of research and was written up in the New York Times. If you plumb the depths here, you’ll find out that about 50 percent of British Atheist gave “fate” as a reason for things, and another 25%, for a total of 75%, said that they battled this idea of fate being real.
I would suggest that this feeling of fate is a piece of wiring that God gave us. Or as Pascal pointed out:
What else does this craving, and this helplessness, proclaim but that there was once in man a true happiness, of which all that now remains is the empty print and trace? This he tries in vain to fill with everything around him, seeking in things that are not there the help he cannot find in those that are, though none can help, since this infinite abyss can be filled only with an infinite and immutable object; in other words by God himself.
[Pascal, Pensees #425]
And this is the one forecast we must get right.