It starts normal. A family has two children.

1. What is the probability that the youngest is a boy?
2. How would your answer change if we know that the family has a boy?
3. How would your answer to part 1. change if we condition on a family that has a boy born on Sunday?

Before answering each let’s figure out the framework (at least for 1. and 2.). The sample space is all (ordered) sequences of length @@@2@@@, consisting of the symbols @@@b@@@ and @@@g@@@, representing the gender of the child, from eldest to youngest. @@@bg@@@, means first born boy and a younger sister. By the product rule, the sample space has @@@2\times 2@@@ elements. Since we did not make any assumptions, we assume that the probability on this space is uniform, and so the probability of every event is the number of its elements divided by @@@4@@@.

1. Let @@@A@@@ be the event “youngest is a boy”, that is @@@\{gb,bb\}@@@. This event has @@@2\times 1@@@ elements, thus its probability is @@@\frac 12@@@. This answers the first question. #This involves conditioning. Let @@@B@@@ be the event “family has a boy”. Then what we’re asking is for the probability of @@@A@@@ conditioned on (or given) @@@B@@@, or, in symbols, @@@P(A|B)@@@. The complement of @@@B@@@ is @@@\{gg\}@@@. Therefore @@@P(B) =1-\frac14 = \frac 34@@@. We need to calculate the ratio @@@P(A\cap B)/ P(B)@@@. Since the statement “youngest is as boy” implies “family has a boy”, it follows that the corresponding events satisfy @@@A\subset B@@@ (if youngest is boy we definitely have a boy). Therefore @@@A\cap B = A@@@ and so @@@P(A\cap B) = P(A)= \frac 12@@@. The conditional probability is therefore

In other words, conditioning eliminates the outcome {::nomarkdown}@@@\{gg\}@@@{:/nomarkdown} from the sample space, leaving us with {::nomarkdown}@@@3@@@{:/nomarkdown} outcomes {::nomarkdown}@@@\{bb,bg,gb\}@@@{:/nomarkdown}, and out of which exactly two correspond to youngest is a boy.  1. This one is a little tricky. Not because of the math, but because the question is ambiguous: some may argue that the additional information on day of birth is completely irrelevant, so the answer is identical to b., while some may argue that this information means that the sample space is different and includes the day on which each child is born. With this latter interpretation the sample space is sequences of length two of symbols of the form {::nomarkdown}@@@x_i@@@{:/nomarkdown}, where {::nomarkdown}@@@x@@@{:/nomarkdown} represents the gender, being equal to {::nomarkdown}@@@b@@@{:/nomarkdown} or {::nomarkdown}@@@g@@@{:/nomarkdown}, while {::nomarkdown}@@@i@@@{:/nomarkdown} represents the day of the week, {::nomarkdown}@@@1@@@{:/nomarkdown} for Sunday, {::nomarkdown}@@@2@@@{:/nomarkdown} for Monday, ..., and {::nomarkdown}@@@7@@@{:/nomarkdown} for Saturday. Thus, by the product rule, we have {::nomarkdown}@@@14=2\times 7@@@{:/nomarkdown} symbols, and we have exactly {::nomarkdown}@@@14^2@@@{:/nomarkdown} sequences of length {::nomarkdown}@@@2@@@{:/nomarkdown} obtained from these symbols. Time for calculations.
- Complement of {::nomarkdown}@@@B@@@{:/nomarkdown} is all sequences without the symbol {::nomarkdown}@@@b_1@@@{:/nomarkdown}. This has {::nomarkdown}@@@13^2@@@{:/nomarkdown} sequences. Therefore {::nomarkdown}@@@|B|= 14^2-13^2 = 27@@@{:/nomarkdown}.
- Number of sequences with youngest being a boy is {::nomarkdown}@@@14*7@@@{:/nomarkdown}. Among those, exactly {::nomarkdown}@@@13*6@@@{:/nomarkdown} do not have a boy born on a Sunday. Therefore, number of elements in {::nomarkdown}@@@A\cap B@@@{:/nomarkdown} is {::nomarkdown}@@@14*7-13*6 = 13+ 7= 20@@@{:/nomarkdown}. 
- Summarizing, the answer is {::nomarkdown}@@@|A\cap B|/|B| = 20/27@@@{:/nomarkdown}. 

Three candidates are running for office. The race is tight, and all outcomes are equally likely. You rank them (winner, second, third).

1. What is the probability that your guess is correct?
2. Assuming that your guess has at least one correct ranking what is the probability your guess is correct?
3. Assuming your guess for the top ranked candidate is correct, what is the probability that your guess is correct?

From the wording of the problem, the underlying probability measure @@@P@@@ is the uniform measure on all different rankings. A ranking is a permutation of the three candidates, therefore @@@P(A)=|A|/3!=|A|/6@@@ for any event @@@A@@@.

Now for the actual probabilities:

1. Let @@@A@@@ be the event that your ranking (for all candidates) is correct. This contains exactly one outcome so @@@P(A) = 1/6@@@, giving us an answer.
2. The additional information in the second part amounts to conditioning on the event that your guess has at least one correct candidate. Call this event @@@B_1@@@. Now the complement of @@@B_1@@@ is all outcomes where every ranking is wrong. There are two ways to guess the first wrong, and each of those corresponds to exactly one way of selecting the second and last wrong. Altogether @@@B_1^c@@@ has two elements, so @@@B_1@@@ has @@@4@@@ elements. Note that @@@A\cap B_1@@@ is @@@A@@@ because @@@A\subset B_1@@@ (if you guessed all correctly, you definitely guessed at least one correctly). Therefore the answer, @@@P(A|B_1)=1/4@@@. The additional information increases your chances of guessing correctly.
3. Finally, let @@@B@@@ be the event that you guessed the winner correctly. @@@B@@@ has two elements, and similarly to the last calculation, @@@A \cap B =A@@@, because @@@A\subset B@@@. Thus, @@@P(A|B)=1/2@@@.

It is clear why @@@P(A)< P(A|B_1)@@@ and @@@P(A)< P(A|B)@@@. But why is the difference between conditioning on the two events @@@B_1@@@ and @@@B@@@? The reason is that @@@B@@@ actually gives us more information than @@@B_1@@@, in the sense that @@@B@@@ limits the number of possibilities to @@@2@@@, while @@@B_1@@@ limits it to twice as much…

Consider the following table (source 1, source 2): comment %} \begin{array} \hline &\mbox{Educational attainment} & \%\mbox{ Unemployment rate } & \%\mbox{ Part of workforce }
\hline\hline 1 &\mbox{Doctoral degree} & 1.6 & 2
\hline 2 &\mbox{Professional degree} &1.6& 2
\hline 3 &\mbox{Master’s degree}& 2.4& 11
\hline 4 &\mbox{Bachelor’s degree} &2.7& 24
\hline 5& \mbox{Associate’s degree} &3.6& 11
\hline 6& \mbox{Some college, no degree}& 4.4& 16
\hline 7& \mbox{High school diploma} &5.2& 26
\hline 8& \mbox{Less than a high school diploma} &7.4 & 8
\hline \end{array} endcomment %} | Educational attainment | % Unemployment | % of work force | | — | — | — | | 1 | Doctoral degree | 1.6 | 2 | | 2 | Professional degree | 1.6 | 2 | | 3 | Master’s degree | 2.4 | 11 | | 4 | Bachelor’s degree | 2.7 | 24 | | 5 | Associate’s degree | 3.6 | 11 | | 6 | Some college, no degree | 4.4 | 16 | | 7 | High school diploma | 5.2 | 26 | | 8 | Less than a high school diploma | 7.4 | 8 |

Let @@@\Omega@@@ denote the entire workforce, and let @@@P@@@ be the uniform probability measure: the probability of an event is its relative frequency.

• Let @@@A@@@ be part of the workforce unemployed at the time the data for the table were collected.
• For each numbered line @@@j@@@, let @@@B_j@@@ the corresponding part of the workforce.

The right column of row @@@j@@@ gives @@@P(B_j)@@@, and the middle column gives the unemployment rate among members of @@@B_j@@@, or @@@P(A|B_j)@@@. If we are asked to find the overall unemployment rate, @@@P(A)@@@, then we can recover it easily from the total probability formula: \begin{align} P(A) &= \sum_{j=1}^8 P(A|B_j) P(B_j)
& = 0.0160.02+0.0160.02+0.024.11\&+0.0270.24+0.0360.11+0.044.16+0.052.26+0.0740.08
&=0.04. \end{align}

Earlier this chapter we quoted the following data. The probability of households with a gun is @@@42\%@@@, the probability of a gun in rural households is @@@58\%@@@ and the probability of a gun in other households (quoted research lists urban, but we wrote “other” to ensure we cover all other than rural) is @@@29\%@@@.

With the given figures, what proportion of households are rural?

To solve, let @@@G@@@ be the event “a gun in household” and @@@R@@@ be the event “rural household”. By total probability formula,

Copying what we are given, we have

Thus, @@@0.13 = 0.29P(R)@@@, or @@@P(R)=44.82\%@@@.

Suppose we were not given the third column, but are given that the unemployment rate is @@@4\%@@@. What can we say about the proportion of the employment force with high-school diploma? Well, it may be as low as zero, that’s clear, but cannot be too large. Why?

Therefore, @@@P(B_7) \le 0.04/0.052 = 0.77.@@@ Note that the first inequality is due to the monotonicity property of probability measures. This estimate is crude (and very far from the actual figure), but at least tells us something.

You’re stuck in a hall in a mine. There are three exists, all look the same. The first leads to a trap and will get you killed. The second will take you out. The third will lead you to a maze that will take you back to the hall. Assuming you select each door with equal probability (each time), and you cannot distinguish between the exists, what is the probability you’ll be able to exit?

Let @@@A@@@ be the event that you will exit. Let @@@B_j@@@, @@@j=1,2,3@@@ be the event that the exit selected first is the @@@j@@@-th. What we’re given is the following:

• @@@P(A|B_1)=0,P(A|B_2)=1,P(A|B_3)=?@@@
• @@@P(B_1)=P(B_2)=P(B_3)=\frac 13@@@

Let’s resolve the question mark. If you take the third exit then you’re eventually starting afresh, so this conditional probability is actually… @@@P(A)@@@. Now use the total probability formula:

So, @@@\frac 23 P(A) = \frac 13@@@, that is @@@P(A)= 1/2@@@.

Let’s think heuristically: we’ll eventually choose one of the first two exists, and that would be the end of the game. By the given assumptions, conditioning on choosing the first or the second, the probability that either one is chosen is the same, @@@\frac 12@@@.

You are about to board a plane. There are @@@n\ge 2@@@ passengers. The first is boarding the plane but lost their boarding pass (or phone) and upon entering the plane, randomly picks a seat. Each of the remaining passengers sit in their designated seat, if available, or randomly pick an available seat otherwise. What is the probability that the last passenger gets to sit in their designated spot?

Let @@@B_j,~j=1,\dots,n@@@ be the events that the first passenger takes the @@@j@@@-th seat. These are disjoint, their union is @@@\Omega@@@ and @@@P(B_j)=\frac 1n@@@. Let @@@A@@@ be the event that the @@@n@@@-th passenger got their designated seat. Knowing that the first sat where they were supposed to implies everyone else sat at their designated spots. So @@@P(A |B_1)=1@@@. Also, @@@P(A |B_n)=0@@@. Make sure you understand why. Therefore, by the total probability formula, we have

It remains to figure out @@@P(A|B_j)@@@. Observe the following:

• When the @@@n@@@-th passenger boards, exactly one seat will be available. That seat can be the @@@n@@@-th seat or the first seat. Why? Every other seat will be taken by one of the previous passengers, either because it was available when they were boarding or because someone else took it.
• Let’s condition on @@@B_j@@@, and take a look at what happens from the time the @@@j@@@-th passenger boards. Conditioned on @@@B_j@@@, all passengers @@@j,j+1,\dots,n-1@@@ treat the @@@n@@@-th and the @@@1@@@-st seat the same way: it is not theirs, so will be equally likely to choose either one if their own seat is taken. Why? Every outcome in @@@A\cap B_j@@@ has a unique corresponding outcome in @@@A^c \cap B_j@@@, simply by exchanging the labels between the @@@1@@@-st and @@@n@@@-th seat. Therefore, @@@P(A^c|B_j) = P(A|B_j)@@@. Since the union of the two events @@@A@@@ and @@@A^c@@@ is the entire sample space, we have that @@@P(A|B_j) = \frac 12@@@. Now plug this into our formula, to obtain

Quite surprising, right?

I’d like to comment that this solution is not the simplest or most general and was given for educational purposes. When the @@@k@@@-th passenger boards, with @@@k\ge 2@@@, the remaining seats are always some subset of @@@n-k+1@@@ seats from the set @@@n-k+2@@@ seats for first passenger, plus those for the remaining @@@n-k+1@@@ passengers (from @@@k@@@-th through last). Each such the possible @@@\binom{n-k+2}{n-k+1}=n-k+2@@@ combination is equally likely as none of previous passengers distinguished any of those seats, and is identified by the unique seat among the @@@n-k+2@@@ not in it. Therefore, the probability that the seat of the @@@k@@@-th passenger is not taken is @@@1-\frac{1}{n-k+2}@@@, the probability of all combinations but the single one without the @@@k@@@-th passenger’s seat.

The setup is the following. Suppose we are looking at admission rates to graduate school of women and men. Our sample space is the set of applicants. Write @@@W@@@ for the event “women” (all women in the pool), @@@M@@@ for the event “men”, and @@@A@@@ for admitted. We are given the admission rates, which are… conditional probabilities: @@@P(A|W)=0.34@@@ and @@@P(A|M)=0.45@@@. Seems like there’s some sort of bias, right?

Now condition further according to departments applied for. For simplicity, assume that there are just two departments, which we will call @@@I@@@ and @@@II@@@. We will assume that each applicant applies to exactly one department. Let’s assume we also know that the admission rates for each department are the following. comment %} [ \begin{array} \hline \mbox{Department} & \%\mbox{ Women applicants admitted} & \%\mbox{ Men Applicants admitted}
\hline\hline I & {\bf 30} & 20
\hline II & {\bf 60} & 50
\hline \end{array} ] endcomment %} | Department | % Women admitted | % Men admitted | | — | — | — | | I | 30 | 20 | | II | 60 | 50 |

Note that for each department the admission rates for women are higher than those for men, yet still, the overall admission rates for women are lower than those for men. How is that even possible? It is, and this is what we call Simpson’s paradox. Let’s write down the entries in the table in terms of conditional probabilities. Let @@@B_I@@@ be the event “applied to department I” and @@@B_{II}@@@ the event “applied to department II”. By assumption, @@@B_{II}=B_I^c@@@. The numbers in the first row are (from left to right) @@@P(A| W \cap B_I)=0.3@@@ (the rate of admitted students among women who applied to department I) and @@@P(A|M \cap B_I)=0.2@@@ (the rate of admitted students among men who applied to department I). By the total probability formula (this is a slight variation, can you notice the difference?)

Dividing by @@@P(W)@@@ we discover

a conditional version of the total probability formula, where @@@P(B_I|W)@@@ is the rate of women who applied to department @@@I@@@. This is a general formula, but with the figures we are given, the following equation needs to be satisfied:

Since @@@B_I^c = B_{II}@@@, it follows that @@@P(B_{II}|W)=1-P(B_I|W)@@@. Therefore we have one unknown here to solve for. After doing some algebra, we obtain @@@0.3 P(B_I | W) = 0.24@@@, or @@@P(B_I | W) = 4/5=80\%@@@.

Repeating the same for @@@M@@@, we obtain

and with the data given:

Again, the only unknown is @@@P(B_I|M)@@@, and after doing the algebra we obtain @@@0.3* P(B_I|M) = 0.05@@@. That is @@@P(B_I|M) = 1/6=16.66..\%@@@.

In summary, although the admission rate for each department was higher for women than for men in every department, the overall admission rate for women was lower than for men. The explanation is simple:

• It is (much) harder to get into department I, just compare second row to first; and
• @@@80\%@@@ of women applied to department I, while only @@@17\%@@@ of men applied to department I.

Police use breathalyzers which display a drunkenness reading if the driver is drunk, and in 5% of the cases when the driver is not drunk. Roughly about one in 1000 drivers are drunk. Suppose a random driver takes the test. What is the probability that the driver is drunk (that is, the “drunk” reading AKA “positive” test result, is not false)?

Let @@@B@@@ be the event that the driver selected is drunk, and let @@@A@@@ be the event that the test displays a a drunkenness reading. We are asked for @@@P(B|A)@@@. We were given three pieces of information: @@@P(A|B)=1@@@, and @@@P(A|B^c)=0.05@@@, and @@@P(B)=0.001@@@. By Bayes’ formula, \begin{align} P(B|A) &= P(A|B) \frac{P(B)}{P(A)} = 1 \times \frac{P(B)}{P(A|B) P(B) + P(A|B^c) P(B^c)}
& = \frac{P(B)}{P(B) + 0.05(1-P(B))}
&=\frac{P(B)}{0.95 P(B) + 0.05}
&=\frac{0.001}{0.0010.95+0.05}= \frac{1}{50.95}<2\%. \end{align*} @@@P(B|A)@@@ is less than fifty times smaller than @@@P(A|B)@@@. This is quite surprising, right?

Let’s forget about the math for the moment, and try to discuss it heuristically, choosing a “perfect sample” of @@@1000@@@ drivers. By perfect, I mean we’re going to taking the proportions literally, namely that exactly one among the @@@1000@@@ is drunk, and exactly @@@5\%@@@ of tests on not-drunk drivers will give a drunk reading. Under these assumptions, we will have exactly @@@1@@@ drunk reading corresponding to an actual drunk driver, and @@@0.05*999=49.95@@@ drunk reading coming from non-drunk drivers. Let’s round the last figure to @@@50@@@. In total, out of the @@@51@@@ drunk readings, only @@@1@@@ came from an actually drunk driver, and therefore the proportion of correct drunk readings among all drunk readings is @@@1/51@@@, which is less than @@@2\%@@@.

The test is not accurate enough to compensate for the fact that the drunk drivers are rare.

IRS audits (I made up all the figures)

• @@@20\%@@@ of returns above @@@\$500K@@@;
• @@@15\%@@@ of returns between @@@\$100K@@@ and @@@\$499K@@@; and
• @@@10\%@@@ of returns below @@@\$100K@@@.

About @@@5\%@@@ of returns are at the first level and @@@60\%@@@ of returns are at the second level. Assuming I was audited, what is the probability I reported an income below @@@\$100K@@@?

Let’s identify the events. @@@A@@@ is the event “being audited”. @@@B_1@@@ is the event “reported less than @@@\$100K@@@”, @@@B_2@@@ is the event “reported between @@@\$100K@@@ and @@@\$499K@@@” and @@@B_3@@@ is the event “reported at least @@@\$500K@@@. We are given the following

• @@@P(A|B_3) = 0.2,~P(A|B_2) = 0.15,P(A|B_1)=0.1@@@
• @@@P(B_3) = 0.05,P(B_2)=0.6,~P(B_1)=?.@@@ We are asked to find @@@P(B_1|A)@@@, the proportion of returns at the lowest level among all returns audited.

Clearly, @@@B_1,B_2@@@ and @@@B_3@@@ are disjoint and their union is the entire sample space (all levels of income). Therefore

giving us

This settles the question mark. We have all we need to apply Bayes’ formula \eqref{eq:gen_bayes}, but let’s do it in two steps, first identifying the denominator. \begin{align} P(A) &= P(A|B_3)P(B_3) +P(A|B_2)P(B_2)+ P(A|B_1)P(B_1) & = 0.20.05+ 0.150.6+0.10.35 =0.135. \end{align*} Therefore,

In other words, about @@@26\%@@@ of audits are for the lowest income level. This is much more than the proportion of audits among the low income level, @@@P(A|B_1)@@@, which was @@@10\%@@@.

If we’re like to find, say, what proportion of audits are on the highest income level, then we already have all the figures:

less than the actual proportion of audits among the high level. Finally if we want to find @@@P(B_2|A)@@@, we… DO not repeat the computation. Because, conditional probability is a probability on its own right, remember? As the union of the disjoint events @@@B_1,B_2,B_3@@@ is the entire sample space, we have

therefore @@@P(B_2|A) = 1- P(B_1|A) - P(B_3|A)=0.77@@@.