I normally avoid discussing potentially divisive material, (especially if it is tech-irrelevant), and I normally avoid going into the nerdy details about math and science and statistics, but this week you are getting a little of both. I was at lunch with friends. A woman begins an anecdote by describing her manager as someone who tries to cook every meal for themselves. The lack of pronoun lingers just long enough for some of us to makes assumptions about the gender of this manager-who-cooks-many-meals. I’ll even give you another sentence here so you can take the time to make some assumptions of your own.
Eventually we learn that he does it as a means to stick to a disciplined diet. My poor roommate admits aloud that he had assumed the manager was a woman. He gets teased in that playful way you would expect: all of the women at the table accuse him of being a little bit sexist for associating “women” and “cooking”. He defends himself by saying that, no, it was just a matter of probability; Considering only the population of people who cook most of their own meals, isn’t it reasonable to assume more of them are female?
The situation is quickly defused by someone else agreeing that he isn’t sexist, he’s just gender frequentist. To me this is about the funniest stats joke I have heard in a while, to the point that yes, I ran all the way here just to tell it to you despite knowing that you would not understand it. I can’t possibly explain the schism between frequentist and bayesian statisticians in a terse way, but it might suffice to say that the frequentist perspective was dominant in the mid-20th century, though might be a little simplistic and old-fashioned in the face of rampant computing technology. The joke runs a lot deeper than that, but basically “gender frequentist” was a clever way to call my roommate old-fashioned without having to muck about with an emotion-laden term like sexism and also recognize that, yes, there are reasonable points of view that support the assumption he made.
I’ve already indulged in some recent math-talk, but the lunch-time banter reminded me of two important things:
1. People use their notion of probability to make decisions all the time.
2. Nobody really agrees on what probability means.
You’d think it would be simple – and it is with simple examples like dice and coins – but what sort of dice are rolled to determine how many car accidents will happen today? How fair is the coin that determines whether or not you will survive a complicated surgery? To drive the point home, my roommate spoke of probability in the above, but what data was he even using? When we talk about the odds of something, what are we really talking about?
We might accept that the layperson’s idea of probability is flawed, but there really isn’t consensus among the experts either.
One notion of probability isn’t really about chance but rather about past observation. If I sample 500 people who cook their own meals, and 400 of them are women and 100 of them are men, then we say the probability of one of them being a woman is 80%. That number isn’t a chance, it’s a summary of actual observations. To wit, this is essentially the frequentist perspective, attempting to avoid all assumptions by relying strictly on describing collected data.
If you change policies and ask yourself, “does this improve the chances that a patient will return?”, how do you answer that question? You probably track the percentage of returning patients overtime and hope to see that figure rise. In this sense you are demonstrating that a probability of something happening is improving, and yet those probabilities are just summary statistics.
In other applications, though, you might want to predict an outcome. In this sense, probability becomes a vague belief. You can say the odds of your favorite team winning a game are 60%, which, when you think about it, is kind of absurd, since the event relies mostly on the decisions of athletes, and not die rolls. The probability is just a notion of how certain you are of the future. This measure might be rigorously justified by past observations, sure, but the point is that you are describing non-random events with probability, which departs greatly from the practice of summarizing the past.
Or you can be like my roommate and “play the odds” despite lacking any formal data collection; again, probability is just a belief, an intuitive way to explain that you have a sliding scale of certainty locked away in your head, and that you calibrate with past observations. If his anecdotal experience lacks rigor, (and it does), we must ask questions about what the “right” way to collect observations is. Answering this becomes less mathematical and more philosophical; What are any observations other than the things you “happened” to observe?
So, you use your sliding belief scale to make decisions, tending toward the things that you believe to be more likely, but there is another step: evaluating the value of each outcome. You don’t drive conservatively because you think you are likely to get into an accident if you don’t, you do it because the cost of getting into an accident is severe and the odds of it happening need to be severely minimized.
The balance between the two judgments, and how well you can make these judgments, characterizes how people make decisions. Someone who is extremely cautious about everything is perhaps making poor cost judgments, evaluating minor mishaps as negatives to always be avoided and so puts the extra effort into certainty. Somebody who is more laid back perhaps balances effort with the knowledge that if something goes wrong, it won’t be so bad. The reckless often underestimate how their actions improve the chances of bad things happening.
The punchline for you is that back in our original story I kept silent, afraid to admit aloud that I had assumed “man” because I heard the word “manager”. I teased you with the scenario because I knew it would confuse your assumptions: Is the manager-who-cooks more likely to be a man because they are a manager or a woman because they cook? This highlights how our intuitive probability-slide-rule breaks down with more variables. Which of those two features is more important? It’s hard to answer without some real analysis. Our fuzzy probability intuition tends to work when the variables are simple and our observations are reliable. If you keep a narrow view of the world, you will make narrow decisions based on poor observations. If you are eager in your decisions, you will miss when the context is more complicated.