In the spirit of the new year, I wrote last week about innovation and how it defies intuition. With the year still pretty new I’d like to keep such lofty ideas in the air. While I’ve waxed philosophic on innovation a few times, there really isn’t some magical strategy that just makes you innovate. In practice, you have to exercise your ability to think differently and to think counterintuitively. But what, specifically, can help you do that?
As I do get the “math and science” bug now and then, I want to go with that. Over the last few years I’ve learned a good deal more math than I used to know, and along the way I became fascinated with a particular law of probabilty. Now, bear in mind that this is a *law*, rigorously proven by math, and cannot be refuted by anyone, ever. It’s called Emmott’s Law and it goes like this: Probability has a 100% chance of being counterintuitive. Nothing separates the mathematicians from the employed quite like probability. So, against my better judgment …
The Buzz: Marilyn vos Savant was wrong that one time.
The Anti-Buzz: No, she wasn’t.
Why: I don’t know why I want to visit the horrors of the Monty Hall problem upon myself.
The summary is that a prize is behind one of three doors, and you choose one, but before you open it Monty Hall floats down from the sky and opens one of the other doors, constrained that he will never open the door with the prize, (that part is important), then offers you the chance to switch to the other unopened door. The question is, does switching doors increase your odds of getting the prize? The solution is that your odds of getting the prize are 2/3 if you switch, and 1/3 if you don’t, so yes.
People hate this claim, but it’s correct. It defies casual intuition, (either door is just a random guess, so they should have the same odds, right?), and people like to think that scientists are crazy and don’t live in the real world, so they are wrong because in the real world doors don’t play tricks like that. In seriousness though, not every one reads Parade magazine, so if the solution bothers you, go read about it.
I’m not here to just explain the most-explained math problem in the past 20 years because it’s already been done, ad naseum, for about 20 years. Instead I want to discuss why the popular perception of this problem, and probability in general, are important.
First, Savant’s famous discussion of the problem was not the first formulation of the problem, nor was she the first to come up with the solution, but it was the first time the problem was widely known, even in academic circles. Savant received some 1,000 letters from real actual mathematicians insisting that she was wrong, (despite publish mathematics papers 15 years prior that demonstrated otherwise). The bitter irony were those attacking her for perpetuating the deplorable state of mathematics education in this country, without realizing exactly how deplorable it really was, (and I’m sure MIT enjoys being the only university to initially accept the solution). If you don’t like this problem, don’t feel bad, because even the smartest of people got it wrong.
Go back to last week when I said that yesterday’s innovation is today’s common sense. I’m working on a PhD of my own, but as of this writing I am maybe only half as educated as the horde of mathematicians who foolishly lambasted Savant 20 years ago, but my story is that the Monty Hall problem is a part of one lecture one day in a sophmore level discrete math class. The eldritch magic that thwarted the great masters in 1990 is just a common mathemagical incantation in 2012, a bone thrown to any 20-year old willing to sit through discrete math. Deplorable mathematics education indeed; the paradigm shifted and we live in a post-Monty Hall world.
However, the world after the storm is fraught with the same problem: people were too accepting of the status quo, and they still are. Now any yahoo knows that when Monty Hall asks you to switch, you say yes, but most of these people couldn’t tell you why. In 1990 Savant was wrong because a bunch of mathematicians said she was and so, by the laws of peer pressure, she was wrong. Now every mathematicians says she is right and so, by the laws of peer pressure, she is right. Both worlds are just worlds where authority is given power when the math gets too hard. The person who reads the Monty Hall solution today and insists that switching doors does not improve your odds of winning is still wrong, but they are also a hero of sorts, because they are actually making an effort to understand the universe better and not simply accept the established wisdom. If they applied that attitude to something else, they might just invent the next tablet PC.
The other interesting discussion is why people so regularly get this question wrong. It raises serious questions about casual intuition about strategy and probability. The most often cited “bad intuitions” are that people tend to assume that all events are uniformly distributed, that all events are independent of each other, and that information does not change probability. I see these assumptions all the time, even among my colleagues.
Most people are used to dice or coins. A die is uniformly distributed, (all sides have equal probability of being rolled), and so is a coin. Die rolls are independent of each other. What you roll on one die does not affect what you roll on the other. Same with coins. Most people understand this too well – we all have that irritating friend who thinks magical fairness ghosts have control of the dice and will make a 5 more likely to come up because it is “due” – but most people have no idea what dependent events or conditional probability look like, (Hint: It looks kind of like Monty Hall floating down from the sky. With diamonds). Unfortunately, your casual interactions with probability set you up for failure. Nothing that matters behaves like dice.
And yes, information does change probability. Probabilities are not decrees from God, they are just calculations made with current information. When the information changes, so does the probability. If I roll a die and ask you for the odds that I rolled a 5, you would say 1/6. Correct. If I told you that the number I rolled was even, now what are the odds I rolled a 5? Zero. I can’t have rolled an even number and a five, it’s impossible. But a moment ago it was 1/6, not zero! I didn’t move the die, I just gave you information! Instead, if I told you that the number I rolled was odd, now what are the odds I rolled a 5? It’s not zero, but it’s not 1/6 either; it’s 1/3. Information changes probability, because probability is just a guess based on what is known.
So my question to you is, how often have your decisions and policies gone unchanged in the face of new information? How often have you overvalued your initial decision, trusted the status quo, and refused to switch doors? How often have you treated new information as irrelevant noise? That’s the lesson you should take from the Monty Hall problem, not the math, but avoiding the angry status quo stampede when you know it is wrong.
If you like this sort of puzzle, I suggest you try the Boy or Girl problem, and if you really want to shatter your intuition in half, google the “blue eyes island” problem, (I have reasons for not linking directly).
In the spirit of counterintuitive logic, I leave you with the best math joke of 2011:
Three logicians walk into a bar. The bartender asks, “do all of you want a drink?” The first logician says “I don’t know,” the second one says “I don’t know,” and the third one says, “yes.”