When common sense is senseless: the Monty Hall Solution

In the previous article, I presented the Monty Hall Problem, a famous problem in elementary probability, based on "Let's make a deal", a television game show of years gone by. If you haven't read that article, what follows is likely to make little sense. It would therefore probably be better to read that article before continuing to read this one.

As a reminder, here is a video clip presenting the game and the problem:



The player has two options: switch to the other closed door or stay with her/his original choice. What should he/she do to maximise her/his chances of winning? There are three possible answers:

1. Switching increases the chances of winning
2. Staying with the original choice increases the chances of winning
3. It makes no difference at all

It turns out that most people think it is a silly question and that option 3, it makes no difference at all, is the correct answer common-sense dictates. If you have chosen this option, you have a lot of friends.

And, you know what? It is indeed the correct answer for some people. So far the good news for common sense.

And now the bad news: while it is the correct answer for some people, it is not the correct answer for you!

Now, think of it. How can this be? This makes no sense at all, does it?

You are left with two choices. One door hides a goat, the other hides a car. If you stay with your original choice, door 3, you will win either a car or a goat. That's one chance in two of winning the car, yes?

Wrong.

Should you switch, you end up with the same possibilities: the door hides either a car or a goat. Again, one chance in two of winning the car, yes?

Wrong.

As a result, you have one chance in two if you stay, and you have one chance in two if you switch. Clearly, option 3, It makes no difference at all, is the correct choice, yes?

Wrong.

As I said, this option is perfectly valid and the right choice for some people, but again, it is not the right choice for you.

Well then, what is the right choice for you?

It turns out that option 1, switching increases the chances of winning, is the right choice for you.

Is your head starting to spin? Take heart, what I said is perfectly true, it isn't complicated and while it isn't common sense, it does make perfect sense.

First of all, let's see what happens if you switch in the example we started in the previous article. The player decides not to switch and stays with her/his first choice:



After confirming the player's choice, the host opens the corresponding door:

Wrong choice. The player did not follow the advice, and lost. The car was behind door 2. See?

Of course, as you have probably guessed, there is more to the story than this. What about an explanation, for example? Who says I did not manipulate the conditions and the player to make it just so? The answer is, of course, that I did indeed manipulate the example to make the point as clearly as possible. However, this does not mean that the advice was wrong.

Let's look at the previous game once more. Remember, first, there are three closed doors. Two conceal a goat, one conceals the coveted car:

The player, who wants to win the car, must now choose one of the three doors:

Hence, the player has one chance in three of choosing the door with the car. One way to look at this problem is to look, not only at the choice one makes, but also at the choices that one does not make:

By selecting door 3, the player also chooses not to select doors 1 and 2. This means that the player has two chances in three of losing the car, i.e. the probability of losing the car is twice as high as the probability of winning the car.

Another way of describing this situation, is that the probability that the car is behind one of the doors in the rectangle, is two in three, whereas the probability that the car is in the stand-alone square is one in three, or half the probability of the rectangle.

But now, the host magnanimously opens one of the two doors in the rectangle to reveal a goat:

Nothing has changed, except that one door which was previously closed, is now open. The likelihood that the car is behind one of the doors in the rectangle is still  two in three. The likelihood that the car is behind door 3 is also still one in three.

But, we now also know that door 1 does not hide the car. This means that the likelihood that the car is behind door 2 is now twice as high as the likelihood that it is behind door 3, since we can safely discard door 1.

In other words, switching is the adviseable option, since that increases one's chances from one in three to two in three. Winning is not guaranteed when switching, however. It is still possible to lose, but it is more likely that one will win.

Very few people are able to find out the correct solution. As I wrote before, the vast majority of people think that switching or staying makes no difference at all. They are wrong, but that is what common sense tells them.

Common sense helps us stay alive in nature, by helping us decide when to run from a lion who wants to eat us, and when not to waste energy running from a full-bellied lion who merely wants to sleep. In more complex situations however, common sense can easily lead us astray.

This problem is a very simple one, yet almost nobody solves it correctly, because it is so counter-intuitive. Welcome to the land of "conditional probabilities". Here, the probability of an event is not only dependent on an existing situation, but also on what happened before.

This is why I wrote that, for some people, switching or not switching would make no difference at all. Who are these people? The people who walked into the room, only to see one open door and two doors closed and who were therefore unaware of what occurred before. For them, the probabilities of having the car behind door 2 or door 3 are equal.

After seeing the above explanation, many people still refuse to accept the solution. Maybe you are one of them (or not). While problems involving probabilities can often be very complex, the Monty Hall problem is not, and this has an advantage. It is actually possible to enumerate every single possible case for this problem.

Consider this: there are three starting possibilities. The car can be behind door 1, door 2 or door 3. Then, the player can select one of those three doors, leading to nine possibilities (three times three). Furthermore, the player can either stay with her/his choice or switch, two possibilities that lead to eighteen possible combinations in total (three times three times two):

If you only count nine lines, you are -of course- correct. However, don't forget that there are two columns: nine switches and nine stays. That makes eighteen combinations in total.

Although it is probably more boring than watching paint dry, it may be interesting to study each of the 18 cases while they are being played out. To this end, please view the video I uploaded to YouTube:



Since the player cannot know whether he/she should switch or stay, the best strategy is to switch, as the table shows. Switching means that the probability of winning is twice the probability of winning when not switching.

It is really hard for humans to wrap their heads around the Monty Hall Problem. This is, however, not necessarily the case for other animals. Pigeons, for example, do a lot better [2].

Conclusion
I wrote this article as a warning.

The Monty Hall Problem shows us that common sense is not to be trusted. Yes, it can lead us to the correct solution when we face a problem. Unfortunately, it can also lead us to the wrong solution. In other words, it makes no sense to trust common sense or "instinct", as the Monty Hall Problem very clearly demonstrates.

Another warning can be seen in the way I treated this problem: a good skeptic always investigates a problem. In spite of popular belief, skepticism is not about denying what others take at face value. Skepticism is about investigating whatever comes our way. Sometimes, this is very easy, such as for the Monty Hall Problem.

Sometimes, it gets a lot more complicated, but whatever the case may be, the skeptic does his utmost to look at a situation from all angles, to investigate all the possibilities and only then does he/she to come to a tentative conclusion that is itself always open for correction, modification or refutation.

The attitude of the skeptic is the attitude of the doubter, the scientist, not that of the quack or the religionist. As a result, the skeptic must be prepared to be ridiculed by the quack and the religionist, for they will use the skeptic's doubts against her/him to fool the public into believing that the skeptic is an idiot, and many people will fall for this.

The skeptic has but one weapon against this: evidence. Quacks often like to compare themselves to Galileo or other famous scientists who were first ridiculed or even put to death for their seemingly preposterous theories. However, they forget one important element.

For Galileo to become world-famous and part of the bedrock of science, he needed one element the quack and the religionist do not have: the evidence to back up the claim, evidence that shows he or she is right.

Quacks and religionists are not interested in such sordid details as "evidence", and that is why they are almost always wrong, often quite terribly so.

The Monty Hall Problem has a very interesting and controversial history. However interesting, the history of the problem is not really relevant to the problem per se and since Wikipedia [1] has an excellent page on this, I will not talk about it here.

Jeffrey S. Rosenthal is one of Canada's best-known experts on probability and statistics. At the time of the fraud scandal at the Ontario Lottery and Gaming corporation, he was often in the news, because he was the expert called in to investigate the then-alleged fraud.

Not only has he written a hilarious and very interesting book, called Struck by Lightning: The Curious World of Probabilities, in which he talks about the Monty Hall Problem, he also has written an article [3] on the problem, which can be downloaded from his personal website.

References

[1] Several authors, Monty Hall Problem, Wikipedia, retrieved 20 February 2012
[2] Walter T. Herbranson, Julia Schroeder, Are Birds Smarter Than Mathematicians? Pigeons (Columba livia) Perform Optimally on a Version of the Monty Hall Dilemma, Journal of Comparative Psychology, 2010, Vol. 124, No. 1, 1-13
[3] Rosenthal, Jeffrey S. (2005a). "Monty Hall, Monty Fall, Monty Crawl".Math Horizons: September issue, 5–7. Online reprint, 2008

----------
Fine print: I have written this article on my personal "authority" as a computer programmer and skeptic. Any errors, inaccuracies, omissions or other flaws are therefore mine, and mine alone. While I have done my best to be as accurate as possible, this article is nevertheless open to correction, modification or refutation. Please do not hesitate to comment or send me an e-mail if you think something is wrong. I will do my utmost to reply to every comment or message within a reasonable amount of time. Please consult a properly qualified professional before using any information in this and any other of my articles.

-----
This page was last updated on 21 February 2012

When common sense is senseless: the Monty Hall Problem

How often have you heard this: "It's just common sense"? Probably far more often than you care to remember. Common sense is often seen as a sort of innate intuition that leads us to the correct conclusions only a moron would deny.

However, as I will show in this and the following article, common sense can also lead us to the wrong conclusions, indicating that common sense is not to be trusted.

Consider the following problem. In this problem, a player is offered the chance of winning a car. The player is presented with three closed doors. One door is hiding a car, the two other doors are hiding a goat. The trick is to select the door hiding the car.

How the game is played can be seen in the following video clip on YouTube:



Now, let's look at each step separately as far as we can. First, the player is confronted with the three closed doors. He or she has no idea where the car is, every door has an equal chance of hiding it.

The player, who has no idea where the car is, selects a door. Unless the player has a preference or is superstitious, this is a completely random choice:

The host of the game, who knows where the car is, must now open one of the two other doors and reveal the goat behind that door. He/she will not reveal the car. This means that the car is either behind the closed door previously chosen by the player or behind the other closed door. The host graciously offers the player a chance to change her/his mind and switch to the other door.

The player has now two options: switch to the other closed door or stay with her/his original choice. What should he/she do to maximise her/his chances of winning? There are three possible answers:

1. Switching increases the chances of winning
2. Staying with the original choice increases the chances of winning
3. It makes no difference at all

Did you figure it out? Write it down, so you remember what you chose. Don't cheat! I'll wait. Answer in the next article...

----------
Fine print: I have written this article on my personal "authority" as a computer programmer and skeptic. Any errors, inaccuracies, omissions or other flaws are therefore mine, and mine alone. While I have done my best to be as accurate as possible, this article is nevertheless open to correction, modification or refutation. Please do not hesitate to comment or send me an e-mail if you think something is wrong. I will do my utmost to reply to every comment or message within a reasonable amount of time. Please consult a properly qualified professional before using any information in this and any other of my articles.

-----
This page was last updated on 21 February 2012

MS Wars - Hope, Science and the Internet

David Suzuki in 2009

I just finished watching the documentary MS Wars: Hope, Science and the Internet, presented by the indomitable David Suzuki in the CBC's "The Nature of Things" series.

I loved this documentary. The issues involved are intense, painful and devastating for patients, and fascinating for science. All the issues were presented in a fair and balanced non-sensationalist and non-conspiratorialist way.

I would have liked to see a more rigorous explanation of *why* anecdotal evidence is so low on the evidential scale, on what the placebo effect really is, since the patients interviewed clearly had not an inkling of understanding, but I also realise that it is darned hard to explain this to the general public. I've been trying for years to do exactly that, and I'm just not succeeding. That pains me, but reality is often painful, and that's not a valid reason for denying it.

While this is not limited to "liberation therapy" specifically, the documentary also clearly showed what the potential is for damage by non-scientific ways of thinking in combination with social media.

"What's the harm?" asks someone. The harm -or the benefit, as the case may be- is, what we will see 20, 30 years or a century from now, when we see how many people have been helped by this non-cure (we already know for a fact that "liberation therapy" is not a cure) and how many have been harmed.

Unfortunately, as the documentary shows, even without knowing the benefits yet, we already know part of the harm: the hopefully temporary derailment of genuine research and the search for therapies that work.

In my opinion, this documentary demonstrates that we need better education. People should learn from the youngest age possible why scientists do what they do and how they do it. These rules have not been created by lazy bureaucrats and selfish politicians who enjoy torturing people.

These rules have been established by scientists to protect themselves against drawing false conclusions and as a way to ensure that science and medicine are not contaminated with the seemingly plausible but utterly false and harmful.

This is the only way to keep the heroic medicine and quackery of the past from harming us again.

While this documentary is necessarily superficial in its explanations because of time constraints, it does a masterful job at pointing out the issues and explaining why they matter while not -even for an instant- forgetting the deeply human aspects of the tragic situations that bring people to irrationality.

The documentary can be watched online on the CBC's website.

-----
This page was last updated on 14 February 2012