In Thinking Fast and Slow, Daniel Kahneman describes an experiment in which students were asked to consider a fictional student called Tom W. They were given the following information:
The following is a personality sketch of Tom W written during Tom’s senior year in high school by a psychologist, on the basis of psychological tests of uncertain validity:
Tom W is of high intelligence, although lacking in true creativity. He has a need for order and clarity, an for neat and tidy systems in which every detail finds its appropriate place. His writing is rather dull and mechanical, occasionally enlivened by somewhat corny puns and flashes of imagination of the sci-fi type. He has a strong drive for competence. He seems to have little feel and little sympathy for other people, and does not enjoy interacting with others. Self-centered, he nonetheless has a deep moral sense.
They were given nine fields of specialization and asked to rank the likelihood that Tom W was a graduate student in each. The punch line was that the fields that most conformed to the stereotypical description of Tom W, computer science and engineering, also had the least number of students overall, while the fields that seemed to least fit the description, like humanities, were more popular.
Given that the description was unreliable (“uncertain validity”), Kahneman argues, responders should have kept their estimates close to the base rate (the relative proportion of students in the various fields) and assigned a higher probability to Tom W being in the more popular fields, despite the description. The students, he claims, made a “mistake” by basing their rankings on the description.
The Cooperative Principle
I didn’t find this entirely convincing at the time, but it wasn’t until I started reading a little about literary theory that I got a vocabulary for expressing why. Social scientists and literary theorists who study communication find a robust “cooperative principle” at work. That is, people who are communicating with each other try to work together to exchange information, and expect others to do the same.
Paul Grice, who named the principle, broke it down into four maxims. Two of those are the Maxim of Relation and the Maxim of Quantity, which state, respectively, that the communication should be relevant and its length should be proportional to its importance. The communication in Kahneman’s study flouts both maxims: the description of Tom W is long and detailed, but it is intended to be irrelevant to the question asked.
Tricking Someone Doesn’t Prove Anything
Consider the following conversation:
Person A: Did you know that Lichtenstein is the smallest country in Europe?
Person B: Really? I thought Monaco was smaller.
Person A: Nope, it’s Lichtenstein.
Person B: I don’t think so.
Person A: I just looked it up. It’s true.
Person B: You seem pretty confident, I guess you must be right.
Person A: So you agree?
Person B: Sure.
Person A: Aha! It’s not!
Would you conclude from this conversation that Person B is ignorant of European geography? Probably not. It seems more reasonable to conclude that Person B is observing standard social conventions, while Person A is not.
In 2008 the wine critic Robin Goldstein perpetrated a hoax on the magazine Wine Spectator, getting them to give an award to a fictitious restaurant that he made up. What this means about the magazine can be interpreted in a variety of ways, but I agree with Stanley Fish: “the moral is that a hoax that is sufficiently and painstakingly elaborated can deceive anyone if the conditions are favorable. This means that the success of a hoax reflects on the skill of the hoaxer and says nothing about the substantive views of those who were fooled by it.”
It seems to me that Kahneman’s experiment has a lot in common with Goldstein’s hoax. The point that was supposed to be proved — in the hoax, that wine magazines are often full of it; in the experiment, that many people are unaware of principles of probability, and those who are often don’t apply them — would be considered by many to be evident with or without an experiment. But the experiment hardly proved it.
In life, as in fiction, descriptions are expected to be relevant and important. Bill James wrote this about the (real) baseball player Hal Chase:
In the spring of 1909 Chase developed smallpox. Some have attributed Chase’s later disrepute to this untimely disfigurement, that he felt cheated of his youth and became bitter and greedy thereafter. Chase, in truth, was greedy and disagreeable before this, but never mind… reading his life as a work of fiction, we see the pox to have been an external manifestation of the rotten pulp at Chase’s center, a clue to the other players and to the readers, if you will, that a wary eye should be kept upon him.
James knows that catching smallpox doesn’t say anything about a person’s character, but within the conventions of fiction, someone’s physical attributes are often reflective of his character. This kind of thinking seeps into real life as well. While it would be foolish to be suspicious of a real person because he once had smallpox, in fiction it would make perfect sense.
Given that the experiment participants were reading a description of a fictional character, they may well have responded according to the conventions of fiction. They thought they were reading a story, but they were really taking a probability test — or so the experimenters thought. By responding according to the conventions of stories, they failed the statistics test.
What if you participated in an experiment with the following questionnaire?
Tom W is a computer science major.
What is the chance that Tom W is a computer science major?
It’s not obvious how to respond. If you encountered this question in a basic probability class, you might assume you were just being asked to demonstrate that you know that 100% or 1 are probabilistic expressions of certainty, and confidently reply 100%.
But if you were taking a more advanced probability class focusing on Bayesian probability, you probably wouldn’t be so sure. Bayesian probability describes how to synthesize multiple pieces of information. You might think you were being asked to estimate the likelihood that a random student named Tom W is a computer science major in light of an unknown person saying that he is. The person’s opinion improves the chance that Tom W is a computer science major, but it doesn’t make it a certainty. It’s hard to know what the right answer is, but it doesn’t seem to be 100%. Indeed, you might be suspicious because the question seems too simple. You might be wary of answering 100% for fear of missing the point and looking foolish.
This highlights an important point: often, answering a question is at least as much about satisfying the questioner as it is about asserting the truth. In this case it becomes a guessing game about what you’re supposed to say. In the Kahneman experiment, the description seems to be leading you to say computer science. You would have to be incredibly paranoid to suspect that you’re being led in that direction in order to prove a point about the sort of mistakes people make when estimating probabilities.
What if in the original experiment, after the description of Tom W, the experiment included a list of the number of students in each major? This wouldn’t provide any new information: the students were already aware of the statistics. But when you include information, you also imply that that information is important. The original study implied, This description is important. You should use it to inform your answer. The study with the statistics explicitly laid out would imply, These statistics important. You should base your answer on them. It seems to me that in the version of the experiment with the statistics laid out, the participants would strongly consider base rate. The conclusion would then not be that people don’t consider base rate, but that they tend to answer based on the information they’re given.
Kahneman’s experiment is not meaningless. It says something about how people judge probability in a certain situation. However, to call the common response a “mistake” is an over-simplification. All of the following seem to me equally plausible interpretations of what this experiment means:
- The conventions of storytelling are different from the methods of probabilistic estimation. Which of those someone applies depends on context.
- It is easy to trick someone, especially if you’re willing to subvert conventions of communication.
- People often say what they think they’re supposed to say, whether or not it’s true.
- People tend to focus on what’s in front of them.
Certainly Kahneman isn’t unaware of those interpretations. In fact, the last is such an important result of his work that he gives it an acronym: WYSIATI for “what you see is all there is.”
In general it seems as though behavioral economics papers spend too much time on math and methods and not enough on assumptions and interpretations. A lot of work goes into demonstrating statistical significance, when often the real argument — the part you could meaningfully disagree with — lies in the interpretation.