Problem 14 - A Modern Introduction to Probability and Statistics: Understanding Why and How

Question

In this exercise we look at a simple example of random variables \(X_{n}\) that have the property that their distributions converge to the distribution of a random variable \(X\) as \(n \rightarrow \infty\), while it is not true that their expectations converge to the expectation of \(X\). Let for \(n=1,2, \ldots\) the random variables \(X_{n}\) be defined by $$ \mathrm{P}\left(X_{n}=0\right)=1-\frac{1}{n} \quad \text { and } \quad \mathrm{P}\left(X_{n}=7 n\right)=\frac{1}{n} $$ a. Let \(X\) be the random variable that is equal to 0 with probability 1 . Show that for all \(a\) the probability mass functions \(p_{X_{n}}(a)\) of the \(X_{n}\) converge to the probability mass function \(p_{X}(a)\) of \(X\) as \(n \rightarrow \infty\). Note that \(\mathrm{E}[X]=0\). b. Show that nonetheless \(\mathrm{E}\left[X_{n}\right]=7\) for all \(n\).

Step-by-Step Solution

Answer

The distributions of \(X_n\) converge to \(X\), but \(\mathrm{E}[X_n] = 7\) differs from \(\mathrm{E}[X] = 0\).

Step 1: Understanding the Convergence

We are given random variables \(X_n\) with probability distributions defined as \( \mathrm{P}\left(X_{n}=0\right)=1-\frac{1}{n} \) and \( \mathrm{P}\left(X_{n}=7 n\right)=\frac{1}{n} \). We need to demonstrate that these distributions converge to the distribution of \(X\), where \(X=0\) with probability 1.

Step 2: Convergence of Distributions

To show convergence, we examine the limit of the probability mass functions \(p_{X_n}(a)\) for any \(a\). For \(a=0\), as \(n \to \infty\), \( \mathrm{P}(X_{n}=0) = 1-\frac{1}{n} \to 1 \), which equals \( \mathrm{P}(X=0) = 1 \). For any \(a eq 0\), \( \mathrm{P}(X_{n} = a) \to 0 \) as \( \mathrm{P}(X = a) = 0 \). Therefore, the probability mass functions \(p_{X_{n}}(a)\) converge to \(p_{X}(a)\).

Step 3: Verification of Convergence

The convergence implies that for any finite \(a\), \( \lim_{n\to\infty} \mathrm{P}(X_n = a) = \mathrm{P}(X = a) \). Since \(X\) only takes the value 0, we see that as \(n\to\infty\), the distribution behavior of \(X_n\) matches that of \(X\).

Step 4: Expectations of Random Variables

To calculate \( \mathrm{E}[X_n] \), we use the law of expectation: \( \mathrm{E}[X_n] = 0\cdot\left(1-\frac{1}{n}\right) + 7n\cdot\frac{1}{n} = 7 \). This calculation shows that \( \mathrm{E}[X_n] = 7 \) regardless of \(n\).

Step 5: Conclusion on Expectation vs. Distribution

Despite the convergence of the distributions of \(X_n\) to \(X\), the expectations do not converge: \( \lim_{n\to\infty} \mathrm{E}[X_n] = 7 eq \mathrm{E}[X] = 0 \). This illustrates how expectations can diverge even if distributions converge.

Key Concepts

Probability Distributions

In probability theory, a distribution describes how the values of a random variable are spread or distributed. Probability distributions can be represented in various ways, such as probability mass functions (PMFs) for discrete random variables or probability density functions (PDFs) for continuous random variables. They help us understand the likelihood of different outcomes.
Distributions can converge, meaning, as you consider larger and larger values of some parameter (like sample size, often represented as \( n \)), the distribution of a sequence of random variables approaches a limiting distribution. In this specific exercise, we see that the random variables \( X_n \) converge in distribution to \( X \), where \( X = 0 \) with probability 1. This means the behavior of the probability distributions of \( X_n \) resembles that of \( X \) more closely as \( n \) increases.
Visualizing distribution convergence can sometimes be tricky. Imagine plotting the PMFs for \( X_n \) for different values of \( n \). You would notice that the portion of probability mass at zero grows as \( n \) increases, showing convergence to the distribution of \( X \).

• Convergence describes the tendency of probability distributions to become increasingly similar.
• \( X_n \rightarrow X \) in distribution implies the distribution of \( X_n \) approaches that of \( X \).

Expectation of Random Variables

Expectation, or expected value, is a fundamental concept in probability that gives a measure of the center or average of a random variable's distribution. It's analogous to a weighted average where each outcome is multiplied by its probability. Calculating the expectation offers insight into the long-run behavior after many trials or observations.
In mathematical terms, for a discrete random variable \( X_n \) with possible outcomes \( x_i \), the expectation \( \mathrm{E}[X_n] \) is computed as \( \sum_{i} x_i \cdot P(X_n = x_i) \). Despite the distributions \( X_n \) converging to that of \( X \), the expectations do not necessarily follow suit, as this exercise elegantly demonstrates.
In the exercise's example, the expectation of \( X_n \) is calculated using the law of expectation resulting in \( 7 \), which remains constant irrespective of \( n \). However, \( \mathrm{E}[X] \) for the limiting distribution is simply 0, since \( X = 0 \) with probability 1.

• Expectation reflects the average or expected value of a random variable's distribution.
• Converging distributions do not imply converging expectations.
• It is crucial to independently verify whether expectations converge just like distributions.

Probability Mass Function

For discrete random variables, the probability mass function (PMF) is a pivotal tool that assigns probabilities to the different possible outcomes. It indicates how probability is distributed across these outcomes for the random variable.
The PMF \( p_{X_n}(a) \) for a given \( a \) gives \( P(X_n = a) \), providing a clear picture of which outcomes are more or less likely. The sum of all probabilities in a PMF always equals 1, ensuring it is a proper probability model.
When studying convergence, PMFs allow us to formalize the idea as they can show the shifting distribution of probability across values as \( n \) tends to infinity. In this exercise, the PMF for \( X_n \) noticeably shifts its probability mass towards the value 0 as \( n \) increases, making it apparent how the distribution converges.

• PMFs allocate probabilities to discrete random variable outcomes.
• A changing PMF illustrates the shifting nature of a distribution.
• Seeing PMF changes can help visualize distribution convergence.