Let’s assume @@@X_1,X_2,\dots@@@ are nonnegative RVs. We know that the expectation of the sum of any finite number of them is equal to the sum of the expectations. What about the expectation of the sum of all of them?

\begin{thm} \label{thm:mon_conv} Let @@@0\le S_1 \le S_2 \le \dots@@@ be RVs and let @@@S_\infty = \lim_{n\to\infty} S_n@@@, which exists in the extended sense, by the monotonicity assumption. Then @@@E[S_\infty] = \lim_{n\to\infty} E[S_n]@@@.

Proof.

From definition,

$$E[S_\infty] = \int_0^\infty P(S_\infty> x) dx = \lim_{M\to\infty} \int_0^M P(S_\infty >x) dx,$$

where the second equality is simply the definition of the improper integral after the first equality sign.

Next, for every @@@x\ge 0@@@, the event @@@\{S_\infty >x\}@@@ is simply @@@\cup_{n=1}^\infty \{S_n > x\}@@@. Indeed, if some @@@S_n >x@@@, then @@@S_\infty>x@@@, and conversely, since @@@S_n \nearrow S_\infty@@@, if @@@S_\infty>x@@@, then @@@S_n >x@@@ for some @@@n@@@.

By continuity of probability with respect to increasing sequences of events, @@@P(S_\infty >x) = \lim_{n\to\infty} P(S_n > x)@@@. Therefore, we can fix some @@@N\in\N@@@, and choose @@@n@@@ be such that

$$P(S_\infty > \frac{ k}{N} M)\le P( S_n > \frac{k}{N} M) + \frac{1}{NM}$$

for @@@k=0,\dots,N@@@. It follows that

\begin{align} \int_0^M P( S_\infty> x) dx &\le \sum_{k=0}^{N-1} \frac{1}{N} P( S_n > \frac{k}{N} M) + \frac{1}{M}
& \le \frac{1}{N} + \sum_{k=1}^{N-1} \int_{\frac{k-1}{N}M}^{\frac{k}{N}M} P(S_n > x) dx + \frac{1}{M}
& \le \frac{1}{N} + \int_0^M P(S_n > x) dx + \frac{1}{M}. \end{align} Bottom line, for all @@@n@@@ large enough, depending only on the choice of @@@M@@@ and @@@N@@@, we have that

$$ \int_0^M P(S_\infty >x) dx \le \frac{1}{N} + \int_0^M P(S_n >x) dx + \frac{1}{M}.$$

By taking @@@n\to\infty@@@ on the righthand side, we have

$$ \int_0^M P(S_\infty > x) dx \le \frac{1}{N} + \lim_{n\to\infty} E[S_n] + \frac{1}{M}.$$

Since the righthand side holds for all @@@N\in\N@@@ and therefore we can take @@@N\to\infty@@@ on the righthand side too.

$$ \int_0^M P(S_\infty >x) dx \le \lim_{n\to\infty} E[S_n] + \frac{1}{M}.$$

Finally, take @@@M\to\infty@@@ on both sides to conclude with

$$E[S_\infty] \le \lim_{n\to\infty} E[S_n]\le E[S_\infty].$$

The result now follows.

■