Problem 9 - Calculus for Biology and Medicine

Question

Suppose \(X_{1}, X_{2}, \ldots, X_{n}\) are i.i.d. with $$ X_{i}=\left\\{\begin{aligned} -1 & \text { with probability } 0.2 \\ 1 & \text { with probability } 0.5 \\ 2 & \text { with probability } 0.3 \end{aligned}\right. $$ What can you say about \(\frac{1}{n} \sum_{i=1}^{n} X_{i}\) as \(n \rightarrow \infty ?\)

Step-by-Step Solution

Answer

As \(n\) approaches infinity, \(\frac{1}{n} \sum_{i=1}^{n} X_{i}\) converges to 0.9.

Step 1: Identify the Expectation

First, we need to find the expected value of the random variable \(X_i\). The expectation \(E[X_i]\) for \(X_i\) is calculated as follows: \(E[X_i] = (-1) \times 0.2 + 1 \times 0.5 + 2 \times 0.3 = -0.2 + 0.5 + 0.6 = 0.9\).

Step 2: Understand the Law of Large Numbers

Since the variables \(X_1, X_2, \ldots, X_n\) are i.i.d., by the Law of Large Numbers, \(\frac{1}{n} \sum_{i=1}^{n} X_{i}\) will converge to \(E[X_i]\) as \(n\) becomes very large. This means that \(\frac{1}{n} \sum_{i=1}^{n} X_{i}\) will tend to \(0.9\) as \(n \to \infty\).

Step 3: Confirm with Probability

By the Weak Law of Large Numbers, for any \(\epsilon > 0\), \(P\left( \left| \frac{1}{n} \sum_{i=1}^{n} X_{i} - 0.9 \right| < \epsilon \right) \to 1\) as \(n \to \infty\). This essentially means that the sample mean \(\frac{1}{n} \sum_{i=1}^{n} X_{i}\) will be arbitrarily close to \(0.9\) with high probability as the number of trials increases.

Key Concepts

Expectation in Probability

In probability theory, the expectation, or expected value, is a crucial concept that provides a measure of the central tendency of a random variable. It's like a weighted average of all possible values that the random variable can take, where each value is weighted according to its probability. For a discrete random variable, the expectation is calculated by multiplying each possible outcome by its probability and then summing all these products. For example, suppose we have a random variable \(X_i\) with three possible outcomes: \(-1\), \(1\), and \(2\), occurring with probabilities \(0.2\), \(0.5\), and \(0.3\) respectively. The expected value \(E[X_i]\) is computed as follows:- \((-1) \times 0.2 = -0.2\)- \(1 \times 0.5 = 0.5\)- \(2 \times 0.3 = 0.6\)Summing these results, we have \(E[X_i] = -0.2 + 0.5 + 0.6 = 0.9\). This expectation gives us a sense of the average result we would expect if the random variable \(X_i\) were to be observed repeatedly many times.

Independent and Identically Distributed Variables

The term "independent and identically distributed" (often abbreviated as i.i.d.) is a key concept in statistics and probability, particularly when working with sequences of random variables. "Independent" means that the occurrence of any specific event or outcome for one variable does not affect the occurrence of an event for another. In other words, knowing the result of one variable doesn't influence the probability of another. "Identically distributed" implies that each random variable has the same probability distribution, meaning they all have the same probability for each outcome. When dealing with i.i.d. variables, the statistical characteristics are consistent across the board.In the context of our problem, the random variables \(X_1, X_2, \dotsc, X_n\) are independent, meaning no variable influences another, and identically distributed, meaning each follows the same probability distribution as specified by the given probabilities of \(-1\), \(1\), and \(2\). This identical distribution ensures that when calculating statistics like means or variances, each variable contributes equally.

Weak Law of Large Numbers

The Weak Law of Large Numbers is a fundamental theorem in probability theory that addresses the behavior of averages of random variables. It states that the sample average converges in probability towards the expected value as the number of observations increases. More formally, for a sequence of i.i.d. random variables \(X_1, X_2, \ldots, X_n\) with expected value \(E[X_i] = \mu\), the law states:\[ P\left( \left| \frac{1}{n} \sum_{i=1}^{n} X_{i} - \mu \right| < \epsilon \right) \to 1 \quad \text{as} \quad n \to \infty \]for any \(\epsilon > 0\). Simply put, this theorem assures us that as more and more observations are collected, the average of these observations will be very close to the expected value with a high probability.In our case, since \(E[X_i] = 0.9\), the weak law implies that the sample mean \(\frac{1}{n} \sum_{i=1}^{n} X_{i}\) will converge to \(0.9\) as \(n\) becomes very large. Thus, regardless of the randomness inherent in each individual observation, the average will stabilize around the expected value with a high degree of certainty as the number of samples grows.