Betting on stuff that matters

Paul Sabin, a Yale historian, has just written a book about a famous wager between Paul Ehrlich and Julian Simon. Ehrlich, a biologist, predicted a looming environmental crisis, while Simon, an economist, believed that human ingenuity would compensate for the drain on natural resources. They put their views to the test with a bet on how the prices of various natural resources would change over ten years. Simon won the bet, causing Ehrlich to completely change his mind and renounce his lifelong advocacy for environmental protection.

No, wait, that second part didn’t happen. Like the guys who “decode” the Bible and predict an apocalypse, Ehrlich didn’t let the unabated existence of the world slow him down. In his support other scientists figured out that he would have won the bet in the majority of decades in the 20th century, so it could be he was right about the general trend and just got unlucky (or lucky, depending on how you look at it).

When it comes to environmental catastrophes, I wonder if a gradual trend is what we should really be looking for, anyway. In gambling we have a term, “fading,” that means avoiding a disaster. If you get all-in with top pair against a flush draw, you’re fading the flush. The fact that we’ve faded an environmental disaster so far doesn’t tell us how lucky we had to get to do so. According to some accounts JFK estimated the chance of nuclear war with the Soviet Union to be at least 50% during his presidency. Certainly, we’ve come close to destroying our whole world in one fell swoop. Every time I turn on public radio I learn about another facet of the environment that’s out of whack – ocean acidity, coral reefs, probably a hundred more I don’t know about. It’s kind of amazing the planet still works as well as it does.

But back to bets. It’s quite unusual for two public figures to bet on something important. During the last presidential election Nate Silver wanted to bet Joe Scarborough that Obama would win. Silver had predicted a comfortable advantage for Obama, while Scarborough opined, “Anybody that thinks that this race is anything but a tossup right now is such an ideologue, they should be kept away from typewriters, computers, laptops and microphones for the next 10 days, because they’re jokes.” Scarborough was apparently operating on the principle that any time there are two possible outcomes, they are equally likely, which works pretty well in PLO, but not so much elsewhere. When Obama won, a parade was thrown for Silver and Scarborough was stoned in the public square.

Oops, that also didn’t happen. Scarborough went right on hosting his political commentary show and no one thought anything of it. What Silver didn’t understand is that Joe Scarborough isn’t in the business of being right. His actual business is harder to pin down. Looking a certain way…sounding a certain way…a bit of trolling now and then. That’s all part of it. But making accurate predictions doesn’t seem to have anything to do with it.

When habitual bettors offer a wager to non-bettors and are refused, they think they’ve proven their point: I put my money where my mouth is and he doesn’t. More likely, they’ve just bullied someone who’s not comfortable throwing money around. But more fundamentally, offering a bet implies that what counts is who’s right, and for most people most of the time, it’s not.

Perhaps I was wrong

Well, it would have to happen the day after I wrote about that, wouldn’t it? It got checked through on the flop and turn and the river brought a straight. Player A bets 50, player B raises to 175, player C cold calls, player A re-raises to 500, player B goes all-in for 1500, player C calls, and player A calls. Players A and B roll the nuts. Player C checks his cards and his face falls.

For my part, I still won’t be raising in this spot, because I think I’m more likely to misread my hand than my opponents. I have some strong suits, but knowing my hand isn’t one of them.

Food for thought on a pointless river

This spot has come up a few times recently and it blows me away every time. It will be a board like ace-king-queen-ten on the river, with no flush or full house possible, so a jack plus any other Broadway card makes the nut straight. Someone bets, someone else raises, the first guy thinks for less than a second and pots (raises the maximum). The other guy always calls and they chop the pot.

If you don’t see anything strange about this you’re not the only one. No one ever bats an eye when this happens. But I’d argue that if you’re thinking about playing the river in a reasonable way,  it’s very strange. On a board like this in PLO there aren’t really any second best hands. Either you have the straight or you don’t. So when the first guy bets he’s saying he has the straight. When the second guy raises he’s saying he definitely has the straight. If you’re the first guy and you’re sitting there with the straight, you can’t really make any more money. Either he has the straight and you’re chopping, or he has a bluff and he’s not putting any more money in. If this spot comes up between two good players, usually the first guy just calls to save time. If you wanted to make more money, I guess you could raise the minimum to represent some sort of crazy re-bluff, but honestly the only way you’re getting more money is if the other guy misread his hand or doesn’t know the rules. I guess you could make the case that if the other guy doesn’t know what he has you might as well make the maximum, so you should pot, but that’s awfully optimistic. In any case, the way some people bet, I can tell their thought process has two parts: 1. I have the nuts. 2. Pot.

There are two good things that can happen when you bet on the river. You can make someone fold a better hand (that’s a bluff) or you can get called by a worse hand (that’s a value bet). In other words, no matter how awesome your hand is, it doesn’t do you much good if you can’t get called by a worse hand. When someone instantly pots in that spot, I can tell they’re only thinking about their own hand – not the opponent’s hand. But what the other guy has matters every bit as much as what you have. Now in this spot it doesn’t really matter – you’re not making any money no matter what you do. But when I see someone do this, I can almost guarantee they’re messing up a lot of other spots that do matter.

Poker and the Law

Last month poker pro Ola “Odd_Oddsen” Amundsgard defeated Norwegian politician Erlend Wiborg in a heads-up Pot Limit Omaha match. Amundsgard offered a 1 million kroner ($170,000) “freeroll” to any member of Norwegian parliament who would take him on in a match. If the member of parliament won, he would get 1 million, but if he lost, he wouldn’t have to pay anything.

In Norway, as in the United States, poker’s designation as a game of skill or game of chance is central to the laws governing it. It is perfectly legal to play games of skill for rewards (as in, for example, the prize for winning a chess tournament), but games of chance are considered gambling and fall under more restrictive legislation. Thus, those wishing to loosen the regulations around poker seek to define it as a game of skill, while those wishing to tighten the regulations define it as a game of chance.

The whole debate is based on a misunderstanding of the relationship between skill and chance. It is simply not true that a game is either a game of skill or a game of chance. In fact, there is not even any relationship between the two. More of one does not imply less of the other. It’s easy to think of games with all sorts of combinations between skill and chance: a lot of both (backgammon), a lot of skill and a little chance (chess), a lot of chance and a little skill (yahtzee), not much of either (tic-tac-toe).

As is the case with so many topics, the public debate has become a cynical rehashing of semantics, while the core issues go unexamined. The whole luck-skill thing is a red herring. The real issue, to my mind, is whether adults can do what they like with their own money. It’s ironic that the United States now restricts online poker more severely than many other countries. Poker was created in America and is, along with baseball, football, and basketball, one of the games we use to define ourselves. Yet alongside our frontier tradition of freedom and rebellion, we have an equally strong Puritanical tradition of repression and prudery. Attempts to legislate the lifestyles of our fellow citizens are nothing new.

Wiborg, Amundsgard’s opponent in the match, comes from the side that wants poker legalized. As such, his competitive interest in the match was divided (not that it would have made any difference). The match was primarily a PR event, of course. It is telling, however, that no politicians on the other side jumped at the opportunity to compete. If they really believed what they say, they would have had a 50/50 chance at 1 million kroner. I doubt the members of Norwegian parliament are so rich that they could casually turn that down.

I don’t think the politicians who want to outlaw poker are particularly interested in luck or skill; they just find poker unsavory. The poker players, for their part, mostly understand that this whole luck-skill thing is pretty silly, but feel they have to play the hand they’re dealt. The role of luck and skill of poker can be summed up quite easily: anyone can win a lucky hand, but in the long run skillful players will tend to win. There’s not much more to be said about it, but the public debate has been mired on this issue for years.

Something happened to me recently that I feel is related to all this somehow. I had been meeting with a chess student at the library for a few weeks, but at the end of one lesson, the librarian told me they have a policy against playing games and I would have to put my chess board away. Indeed, rule 11 of the Ann Arbor District Library Rules of Behavior “prohibits board games, gambling, card playing, or other games of chance or skill on Library property, except patrons may play board games when such games are provided by the Library as part of an organized activity.” At the very moment I was warned about my chess board, a crazy guy was ranting about how he’d like to kill Obama. You can’t make this stuff up. Naturally, he wasn’t asked to leave. But the guy with the chess board — that’s something that has to be dealt with right away.

Apart from the general weirdness of outlawing chess in a library, the phrase “games of chance or skill” strikes me as particularly odd. Does that leave any games out? Why not just say, simply, “games”? I suppose the rules were drafted with the help of lawyers and they advised throwing that in there.

In the case of poker, at least, it seems as though the law was drafted by people with no particular knowledge of the subject. With poker and many other subjects, if we frame the debate in the terms of the law, we risk coming to some very silly conclusions.


Dots for fun and profit

Dots is a simple game for two players. You make a grid of dots on a piece of paper (we used to play with a roughly ten-by-ten grid, although there’s no rule about how big it should be). The players take turns drawing a line connecting two dots. If you complete a square, you put your initial in the middle to mark it as yours, and you get to go again. The game ends when the whole grid is filled with lines. The player who owns the most squares wins.

At first, it can be difficult to see which moves allow your opponent to finish a square. Thus, some games are decided by simple one-move oversights. But with a few games under your belt it’s relatively easy to avoid those mistakes. If both players are at that level, the game comes down to who gets the biggest runs of squares, more or less at random. There is, however, one trick that will let you win just about every game if you know it and your opponent doesn’t.

Typically the board develops in such a way that there are big clumps of almost-finished squares. If you can claim one of them, you get the rest in a domino effect. The natural play is to take all the available squares, then look for the least damaging move to pass it back to your opponent. The trick is to not take all the squares. You can leave the last two for your opponent. They get those, but then they’ll have to give you the next big clump and you can repeat the process, claiming all the big tracts and giving away just two piddling squares in return.

Of course, you only get to do this once before your opponent realizes what’s up and uses the same approach in future games. At that point the game just comes down to whoever’s move it is when the good moves run out, which is awfully hard to control.

Which, if you think about it, is kind of weird. Ties are rare in dots. In fact, depending on the size of the grid, a tie might be impossible. There’s no hidden information. So at any point in the game there should be a “correct” result: one side or the other could force a win with perfect play. On the very first line, there are probably some moves that win and some moves that lose. But how do you know which are which? There doesn’t seem to be any way to know, or even to have a better chance of being right. In dots the early part of the game is a tedious formality where both players go according to their whims until the rubber hits the road in the late game. Many of the moves are probably horrible mistakes, but there’s no way to know.

Chess and Go, like dots, are too complex to be solved, but they offer players more satisfying ways to guess at good moves. A good player can often tell who’s winning with just a glance at the position. Some formations of pieces work better than others and, to an expert, a good position “looks good.” In dots, a good position is just one move away from a bad position (if the other side were to move, the result would be reversed). It doesn’t seem possible to develop any sort of feel or intuition, as one can in chess or Go.

As such, dots has little to teach us as game players, except that knowing a simple trick can give us a huge advantage, even if we don’t understand the game deeply. But the edge we get from these tricks is likely to disappear quickly if our opponents are smart and observant.

From the game design perspective dots is more interesting. It shows that for a game to be satisfying, it must be too complex to solve definitively, but still admit of shortcuts, guesses, or heuristics. The players can’t know for sure what the best moves are, or the game would get stale, but they must have ways of approaching good moves, or feeling that they do. We have to feel like we’re getting somewhere, but if we ever get there, the game stops being fun.

Tagged , ,


Tyler Cowen at The Marginal Revolution attributes Magnus Carlsen’s ascendance to chess world champion to his “nettlesomemess.” This doesn’t mean he picks his nose or makes annoying sounds while he plays. Rather, it means he plays moves that are tricky to answer correctly.

Carlsen has the highest nettlesomeness score by this metric, because his creative moves pressure the other player and open up a lot of room for mistakes.  In contrast, a player such as Kramnik plays a high percentage of very accurate moves, and of course he is very strong, but those moves are in some way calmer and they are less likely to induce mistakes in response.

Game theory has for the most part ignored chess, focusing as it does on situations where what you do depends on what the other guy does. In chess, in theory at least, you could just survey the position and play the best move without a care in the world for what your opponent might be thinking. According to this account, all the judgment and calculation that goes into making a chess move is just a technical process best left to the experts.

I’ve noticed that some game theorists seem to take a sort of pride in being uninterested in chess (and other games of complete information, like Go). If game theory doesn’t have anything interesting to say about these games, that’s more like a blind spot than a triumph. If we want to learn something meaningful about real world decision making, the process of figuring these things out certainly matters.

I’m excited about the idea of nettlesomeness as it applies to chess, as well as poker, which game theory has historically had more to say about. But I’ve been thinking about nettlesomeness as it applies to a less heralded game: tic-tac-toe.

If both sides play well, a game of tic-tac-toe should end in a tie. This is true no matter how Xs starts the game. No opening move is so good that it forces a win, nor is any so bad that it leads to a loss. From the traditional game theory perspective then all the moves are equally good. Xs can, however, control Os’ available paths to a tie. This is where nettlesomeness comes in.

The most “natural” first move is in the center. After that, Os has eight moves, although it’s effectively two moves: he can go in the corner or the edge. It turns out that going on the edge loses, while going in the corner ties.

ttt center

Xs can also start on the edge. In that case Os can tie by going in the center, any corner, or the opposite edge. The only moves that lose are the edges next to the initial X.

ttt edge

Finally Xs can open in the corner. Then Os must go in the center to hold a tie. All other moves allow Xs to force a win.

ttt corner

This analysis suggests a definition of nettlesomeness.

Nettlesomeness = Losing moves / Possible moves

Let’s call this measure N. Going in the center has an N of .5, the edge has an N of .25, and the corner has an N of .875. According to this metric, even though the center seems to be the natural move, the corner is actually a much better move.

It’s far from a perfect measure of course. If our opponent agrees that going in the center is the most natural move, that’s exactly what he’s likely to do when we go in a corner. Even though he has only one move to save the game, it’s an obvious move. And since it’s tic-tac-toe we can hardly expect to win anyway unless we’re playing a child or an animal. But in more challenging games like chess, this principle – not only fewer saving moves, but less obvious saving moves – is a big deal.

Extra: if you want to work on your tic-tac-toe game xkcd has a nice map.

Edit: fixed edge diagram.

Tagged , ,

The Haig Plan

“Doing precisely what we’ve done 18 times before is exactly the last thing they’ll expect us to do this time!”

Here’s a very general question: if someone did A last time, are they more likely to do A or not-A this time?

How you answer this question will have a lot to say about how you play poker. If someone just showed down a big bluff and they’re betting into you again, do you think they’re more or less likely to be bluffing this time? Do you call or fold?

The question depends on what they think of you, what you think of them, and what you believe about human nature in general.

If you take a generally accepted view of empirical evidence, it seems the more times someone has done A, the more likely they are to do A in the future. Football teams that have won a lot of games in the past are expected to keep winning. The value of the stock market is expected to keep rising because it has risen in the past. If someone we know has behaved rudely in the past, we expect him to behave rudely in the future.

In poker it becomes more complicated because our opponents are incentivized to surprise us and confound our expectations. A football team tries to win every time, but poker players (some of them, anyway) try to trip us up by reversing course. How to balance observed tendencies against the desire to reverse those tendencies?

If our opponents were machines that played the same way every time, their past behavior would indeed be good evidence of how they would likely behave in the future. Not perfect evidence, but in general, the more we saw them do something, the more confident we would be that they’d keep doing it. But since we’re dealing with people, it depends on how they think.

Experiments show that people are bad at simulating random sequences. Specifically, they discount the possibility of long runs too much. So people asked to simulate a random series of coin tosses produce too many sequences like this:


and not enough like this:


Long runs like the second sequence show up more often in actual random trials than in people’s attempts to simulate randomness.

Many players have a sense that they want their bluffs to simulate strong hands, which show up randomly. Therefore they time their bluffs to look like the first sequence of heads and tails above: spread out fairly evenly. If they haven’t raised in awhile, they think the time is right for a bluff; and if they’ve been raising a lot (probably because they’ve been getting good hands) they won’t bluff. This actually makes them less random. The strategy of bluffing sporadically works well not because it imitates randomness, but because it imitates most people’s idea of randomness. As long as you only raise occasionally, most people will give you credit for having a good hand.

I’ve seen a scenario play out like this many times: all of a sudden someone who’s been playing tight for a long time raises three hands in a row. Often when this happens, someone will re-raise him the third time, thinking he can’t have a strong hand three times in a row. This rarely works out for the re-raiser. If someone’s been playing tight for a long time, he probably really is a tight player. He could be a loose player with a truly terrible run of bad cards, but it’s unlikely; and if he were really loose, he wouldn’t let bad cards stop him anyway. What’s more likely is he just got dealt a strong hand three times in a row. It can happen… more often, as the experiments show, than we think it should.

In fact, if he was getting out of line, it was probably on the first raise, not the third one. Maybe he thought all his folding bought him some credibility and he could sneak one by. But the second time? A tight player wouldn’t try his luck again. He’s trying to simulate the flow of strong hands, which he doesn’t imagine would show up twice in a row. The third time is probably the strongest hand of all. By now he’s probably feeling somewhat sheepish about all this raising, so he must have a hand so strong that he has no choice but to raise. This is the worst time to go after him.

This situation taps into what’s known as the gambler’s fallacy, which is basically the belief that things balance themselves out. For example, someone who falls prey to the gambler’s fallacy might think that if a coin has come up heads three times in a row, it’s more likely to be tails the next time. In reality as long as it’s a fair coin it’s still 50/50 to be heads or tails the next time, no matter what came before. While in the long run you’ll tend to get a roughly equal number of heads and tails, there is no force that makes sure things balance out in the short term.

In poker, the chance of being dealt pocket aces is one in 221. The chance of being dealt aces twice in a row would be one out of 221 * 221, which comes out to one in 48841. Very unlikely! But if someone got aces last hand, they are no more or less likely than anyone else at the table to have aces again. Everyone’s looking at the same one in 221. Sometimes someone will justify a play by saying, “How likely was he to have it twice in a row?!” As far as the probabilities go, the previous time has nothing to do with it, although it will certainly affect both players psychologically.

What if you’re playing someone who knows all that? Then it might be a good idea to bluff immediately after you raise. That’s the time most people are least likely to bluff, so if your opponent knows that, he’ll give you a lot of credit for having a good hand. In fact, I’ve seen a video in which a strong player raised twice in a row against another strong player using exactly that reasoning.

But what if we’ve both read the preceding paragraph? Well it’s clear that now we’re getting into a deadly guessing game. Poker players call this dynamic “leveling”: we’re each trying to guess which level the other is on, and each level deeper we go the best strategy flips. Some players try to opt out of this kind of guessing game by basing their plays on ranges. A range is the set of hands with which you perform a certain action.  So maybe I re-raise with pocket jacks or better and ace-king, but also seven-six suited. This way my “bluffs” (seven-six) show up truly randomly, not just my imitation of randomly. Most people associate “playing your cards” with not bluffing, but you can create a strategy based on your cards to bluff at a pre-determined frequency. If you fear that your opponent is at least as sharp as you are, you may want to play this way, but if you consider yourself sharper, you might prefer to play the guessing game.

Some players are obsessed with establishing tendencies for the purpose of reversing them. I associate this strategy with former Michigan football offensive coordinator Mike Debord. Debord would establish a tendency all season — for example, every time he sent the tight end in motion to the left, he’d run left — only to break it in a key game, usually against Ohio State. This sometimes confused the defense, but they usually adapted pretty quickly. In fact, I suspect opposing defensive coordinators picked up on Debord’s meta-tendency of grandly rehearsed reversals to the point where they had a pretty good idea when they were coming.

As with many things in poker, it makes a big difference what your opponent is aware of, and the tendencies they don’t even know about themselves are usually the most reliable. Those are the ones that they’d never think about reversing. So if someone showed a bluff after taking a very strange line in a huge pot, the next time he takes that line, he probably has the nuts. Unless he’s a moron, or he thinks you’ll think that (but now we’re into the leveling war again). But someone might not bluff raise often enough on the flop and not even be aware of it just because he doesn’t know that’s something he’s supposed to do. In that case he’ll never change his tendency unless something forces him to change how he thinks about the game.

Some people imagine that a “wild man” strategy of raising, calling, or folding essentially at random would be hard to play against, but such a strategy usually ends up being extremely unbalanced (way too many bluffs, or not enough). I’ve heard people say, “How can you know what I’m doing? I don’t even know what I’m doing?” Everyone I’ve ever heard say this had strong and exploitable tendencies. In fact, to be effectively unpredictable requires a lot of work and planning. You have to be very intimate with your own ranges to know how often you ought to be bluffing and with which hands.

As to the original question — if someone did A last time, are they more likely to do A or not-A this time? — it’s hard to offer a better answer than “it depends.” In general, most people pay too much attention to short-term history and not enough to long-term tendencies. Dramatic events in recent history are the ones your opponent is most likely to remember and try to confound, while long-term tendencies are less salient and he may not even be aware of them. Perhaps it’s helpful to think of your opponent as a snowboarder: he can do a lot of tricks, twists, and jumps along the way, but eventually he ends up at the bottom of the mountain.

The Detroit School of PLO

Did you know there’s a group of young firebrands from Detroit turning the poker world on end with their unorthodox play? Not content to accept strategies that have proven profitable in the post, they’ve rebuilt the game from the ground up. These are the tenets of the Detroit School of PLO:

  • Nearly any hand is worth a call pre-flop. The worse your hand, the more deceptive when you hit!
  • Counter-intuitively, bad hands get better the more money goes in. You don’t have to worry about your set or flush draw being dominated if you’ve already committed a third of your stack.
  • Pocket trips are worth playing for blockers. If you flop blockers, you’ll have three of them! If you’re lucky enough to get dealt quads all the better.
  • Nothing beats a min-three bet in position. If they four bet, you know they have aces. No matter what happens, you’ll certainly annoy your opponent. Some particularly progressive players have taken this thinking one step further and started min-three betting out of position.
  • It’s all about the nuts. If you don’t have the nuts, you can’t value bet. If your opponent doesn’t have the nuts, he can’t call a big bet.
  • Folding is for sissies.

Think long, think wrong?

When a poker player thinks for a long time, he often makes a mistake. The phenomenon is so common that it has been reduced to a maxim: thing long, thing wrong.

While poker could be accused of having an anti-intellectual atmosphere — some poker players are opposed to thinking, long, short, or otherwise — chess couldn’t really; but even chess players notice a correlation between a long think and a blunder.

It’s easy to tell a story about why this happens: when you think for a long time you become entangled in irrelevant complexities, confuse yourself, and ignore your initial instincts. It would seem to follow that you can improve your performance by forcing yourself to play faster.

This all assumes that the long think is causing the mistake. There’s little doubt that in chess and poker there’s a correlation between thinking for a long time and making a mistake, but, as nerds are constantly reminding us, correlation is not causation. When two things are correlated, either one could be causing the other. In this case though it seems impossible for the mistake to cause the delay, because the mistake happens afterwards; so it must be the delay causing the mistake. But there’s a third possibility that’s often forgotten, that something else causes both of them.

Once you remember that possibility, it’s pretty obvious what’s going on. Not knowing the best play causes both delays, because you’re trying to figure it out, and mistakes, because you still don’t know what to do. While thinking for a long time and making a mistake are correlated, you probably can’t improve your performance by forcing yourself to play faster. What’s really causing the mistake is not knowing the right approach in a certain situation, and playing faster won’t help that. If you want to improve your performance, you need to become more skillful so that the situations where you don’t know what to do come up less often.

In Thinking Fast and Slow, Daniel Kahneman describes an experiment in which students were asked to consider a fictional student called Tom W. They were given the following information:

The following is a personality sketch of Tom W written during Tom’s senior year in high school by a psychologist, on the basis of psychological tests of uncertain validity:

Tom W is of high intelligence, although lacking in true creativity. He has a need for order and clarity, an for neat and tidy systems in which every detail finds its appropriate place. His writing is rather dull and mechanical, occasionally enlivened by somewhat corny puns and flashes of imagination of the sci-fi type. He has a strong drive for competence. He seems to have little feel and little sympathy for other people, and does not enjoy interacting with others. Self-centered, he nonetheless has a deep moral sense.

They were given nine fields of specialization and asked to rank the likelihood that Tom W was a graduate student in each. The punch line was that the fields that most conformed to the stereotypical description of Tom W, computer science and engineering, also had the least number of students overall, while the fields that seemed to least fit the description, like humanities, were more popular.

Given that the description was unreliable (“uncertain validity”), Kahneman argues, responders should have kept their estimates close to the base rate (the relative proportion of students in the various fields) and assigned a higher probability to Tom W being in the more popular fields, despite the description. The students, he claims, made a “mistake” by basing their rankings on the description.

The Cooperative Principle

I didn’t find this entirely convincing at the time, but it wasn’t until I started reading a little about literary theory that I got a vocabulary for expressing why. Social scientists and literary theorists who study communication find a robust “cooperative principle” at work. That is, people who are communicating with each other try to work together to exchange information, and expect others to do the same.

Paul Grice, who named the principle, broke it down into four maxims. Two of those are the Maxim of Relation and the Maxim of Quantity, which state, respectively, that the communication should be relevant and its length should be proportional to its importance. The communication in Kahneman’s study flouts both maxims: the description of Tom W is long and detailed, but it is intended to be irrelevant to the question asked.

Tricking Someone Doesn’t Prove Anything

Consider the following conversation:

Person A: Did you know that Lichtenstein is the smallest country in Europe?

Person B: Really? I thought Monaco was smaller.

Person A: Nope, it’s Lichtenstein.

Person B: I don’t think so.

Person A: I just looked it up. It’s true.

Person B: You seem pretty confident, I guess you must be right.

Person A: So you agree?

Person B: Sure.

Person A: Aha! It’s not!

Would you conclude from this conversation that Person B is ignorant of European geography? Probably not. It seems more reasonable to conclude that Person B is observing standard social conventions, while Person A is not.

In 2008 the wine critic Robin Goldstein perpetrated a hoax on the magazine Wine Spectator, getting them to give an award to a fictitious restaurant that he made up. What this means about the magazine can be interpreted in a variety of ways, but I agree with Stanley Fish: “the moral is that a hoax that is sufficiently and painstakingly elaborated can deceive anyone if the conditions are favorable. This means that the success of a hoax reflects on the skill of the hoaxer and says nothing about the substantive views of those who were fooled by it.”

It seems to me that Kahneman’s experiment has a lot in common with Goldstein’s hoax. The point that was supposed to be proved — in the hoax, that wine magazines are often full of it; in the experiment, that many people are unaware of principles of probability, and those who are often don’t apply them — would be considered by many to be evident with or without an experiment. But the experiment hardly proved it.


In life, as in fiction, descriptions are expected to be relevant and important. Bill James wrote this about the (real) baseball player Hal Chase:

In the spring of 1909 Chase developed smallpox. Some have attributed Chase’s later disrepute to this untimely disfigurement, that he felt cheated of his youth and became bitter and greedy thereafter. Chase, in truth, was greedy and disagreeable before this, but never mind… reading his life as a work of fiction, we see the pox to have been an external manifestation of the rotten pulp at Chase’s center, a clue to the other players and to the readers, if you will, that a wary eye should be kept upon him.

James knows that catching smallpox doesn’t say anything about a person’s character, but within the conventions of fiction, someone’s physical attributes are often reflective of his character. This kind of thinking seeps into real life as well. While it would be foolish to be suspicious of a real person because he once had smallpox, in fiction it would make perfect sense.

Given that the experiment participants were reading a description of a fictional character, they may well have responded according to the conventions of fiction. They thought they were reading a story, but they were really taking a probability test — or so the experimenters thought. By responding according to the conventions of stories, they failed the statistics test.

Context Matters

What if you participated in an experiment with the following questionnaire?

Tom W is a computer science major.

What is the chance that Tom W is a computer science major?

It’s not obvious how to respond. If you encountered this question in a basic probability class, you might assume you were just being asked to demonstrate that you know that 100% or 1 are probabilistic expressions of certainty, and confidently reply 100%.

But if you were taking a more advanced probability class focusing on Bayesian probability, you probably wouldn’t be so sure. Bayesian probability describes how to synthesize multiple pieces of information. You might think you were being asked to estimate the likelihood that a random student named Tom W is a computer science major in light of an unknown person saying that he is. The person’s opinion improves the chance that Tom W is a computer science major, but it doesn’t make it a certainty. It’s hard to know what the right answer is, but it doesn’t seem to be 100%. Indeed, you might be suspicious because the question seems too simple. You might be wary of answering 100% for fear of missing the point and looking foolish.

This highlights an important point: often, answering a question is at least as much about satisfying the questioner as it is about asserting the truth. In this case it becomes a guessing game about what you’re supposed to say. In the Kahneman experiment, the description seems to be leading you to say computer science. You would have to be incredibly paranoid to suspect that you’re being led in that direction in order to prove a point about the sort of mistakes people make when estimating probabilities.

What if in the original experiment, after the description of Tom W, the experiment included a list of the number of students in each major? This wouldn’t provide any new information: the students were already aware of the statistics. But when you include information, you also imply that that information is important. The original study implied, This description is important. You should use it to inform your answer. The study with the statistics explicitly laid out would imply, These statistics important. You should base your answer on them. It seems to me that in the version of the experiment with the statistics laid out, the participants would strongly consider base rate. The conclusion would then not be that people don’t consider base rate, but that they tend to answer based on the information they’re given.


Kahneman’s experiment is not meaningless. It says something about how people judge probability in a certain situation. However, to call the common response a “mistake” is an over-simplification. All of the following seem to me equally plausible interpretations of what this experiment means:

  • The conventions of storytelling are different from the methods of probabilistic estimation. Which of those someone applies depends on context.
  • It is easy to trick someone, especially if you’re willing to subvert conventions of communication.
  • People often say what they think they’re supposed to say, whether or not it’s true.
  • People tend to focus on what’s in front of them.

Certainly Kahneman isn’t unaware of those interpretations. In fact, the last is such an important result of his work that he gives it an acronym: WYSIATI for “what you see is all there is.”

In general it seems as though behavioral economics papers spend too much time on math and methods and not enough on assumptions and interpretations. A lot of work goes into demonstrating statistical significance, when often the real argument — the part you could meaningfully disagree with — lies in the interpretation.

A variety of interpretions


Get every new post delivered to your Inbox.

Join 456 other followers