Conditional probability has numerous applications in scientific, medical and legal reasoning, statistical genetics, cryptography etc. Whenever we observe new evidence or information, the odds of an event must be updated.
A motivating example.
Example. The Monty Hall problem.
On the game show Let’s Make a Deal, hosted by Monty Hall, there are three doors, randomly one of the doors has a car behind them, the other two doors have goats behind them. You as the contestant have no idea, which one has the car, they are all equally likely. But Monty Hall knows which one has the car.
You as a contestant, pick a door, say door \(1\). Then, what happens is, Monty opens up either of the remaining doors \(2\) or \(3\), revealing a goat. The door he opens always has a goat behind it (he never reveals the car!), for example, Monty opens goat door \(3\). So, then you know, that the car is behind the door you initially picked, door \(1\) or the other unopened door \(2\).
Monty then offers the contestant the option of switching to the other unopened door, or keeping your original choice. The question is, should you stay with with your initial choice or should you switch?
Assumption. Monty always opens a goat door. If he has a choice, of which door to open, he picks with equal probabilities.
For example, if you initially guessed right, so the car is behind the door \(1\), doors \(2\) and \(3\) both have goats. Monty could open either door \(2\) or door \(3\). Assume these are equally likely.
Should you stay with your initial choice or switch?
Many people, upon seeing the problem for the first time, argue that there is no advantage to switching : “There are two doors remaining, and one of them has the car, so the chances are 5050”. Controversy has raged about this problem for years and years.
Solution.
Under the standard assumptions above, the answer is, you should switch. Let’s label the doors \(1\) through \(3\). We know that, any of the \(3\) doors are equally likely to be the cardoor.
\(P(C_{1})=P(C_{2})=P(C_{3})=1/3\)
Without the loss of generality, we can assume that the contestant picked the door \(1\). Suppose that Monty opens door \(3\). Suppose we believe the car is behind door \(2\), denoted by event \(C_{2}\). The prior probability that the car is behind door \(2\), \(P(C_{2})=1/3\) can be updated in light of the new information about Monty opening door \(3\).
Bayes’s rule is used to update prior probabilities by incorporating new information.
\(\begin{align}
P(C_{2}M_{3})&=P(C_{2})\cdot\frac{P(M_{3}C_{2})}{P(M_{3})}\\
&=(1/3)\cdot\frac{1}{(1/2)}\\
&=2/3
\end{align}\)
And, the probability that the car is behind door \(1\) is,
\(P(C_{1}M_{3})=1P(C_{2}M_{3})=1/3\)
It would be an abuse of the naive definition of probability, if you just immediately say, that the two doors are equally likely to have a car behind them. There is a \(1/3\)rd chance that the car is behind the first door, and a \(2/3\)rds chance it is behind the second door.
Thus, the observation that Monty opened door \(3\) makes us more sure of our belief that the car is behind door \(2\).
Building correct intuition
Let’s consider an extreme case. Suppose that there are a thousand doors, \(999\) of which contain goats and \(1\) of which has a car. After the contestant’s initial pick, Monty opens opens \(998\) doors with goats behind them. Let’s update our prior probabilities as Monty opens doors with goats behind them.
Assume, Monty opens door \(1000\).
\(
P(C_{2}M_{1000})=P(C_{2})\frac{P(M_{1000}C_{2})}{P(M_{1000})}=\frac{999}{998}\cdot\frac{1}{1000}
\)
Next, Monty opens door \(999\).
\(
P(C_{2}M_{999}M_{1000})=P(C_{2}M_{1000})\frac{P(M_{999}C_{2}M_{1000})}{P(M_{999}M_{1000})}=\frac{999}{997}\cdot\frac{1}{1000}
\)
Continuing in this fashion,
\(P(C_{2}M_{3}\ldots{M_{1000}})=\frac{999}{1000}\)
In this extreme case, it becomes clear that the probabilities are not \(5050\). As Monty eliminates \(998\) of the \(999\) doors, we are extremely confident of the belief that the car is behind the remaining unopened door.
R Simulation
A simple R simulation demonstrates that the strategy to always switch has a success probability of \(2/3\).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

n < 3 trial < function(n,game_result_flag){ doors < 1:n # Contestant's initial choice choice < sample(doors,1,replace=TRUE) # The car is behind one of n doors car_door < sample(doors,1,replace=TRUE) # Monty knows the car door. He randomly opens the goat doors, # but it can't be the goat door or the player's door if(choice != car_door) remaining_doors < doors[c(choice,car_door)] else remaining_doors < doors[c(choice)] # After Monty has opened all of the goat doors, he offers the contestant # the option of switching to the other unopened door. switch_flag < TRUE if(switch_flag == TRUE) { if(choice != car_door) game_result_flag < TRUE else game_result_flag < FALSE }else{ if(choice == car_door) game_result_flag < TRUE else game_result_flag < FALSE } } 
The experiment is repeated \(1\) millon times.

nSim < 10^6 list_of_results < replicate(nSim,trial(n=3,game_result_flag = NULL)) games_won < sum(list_of_results == TRUE) games_lost < nSim  games_won 
The number of games won and lost are :

> games_won [1] 667414 > games_lost [1] 332586 
The links to my notes and the solved problems are given below.
Conditioning and independence
Sequential Sampling
Solved examples