Problem 9 - Calculus for Biology and Medicine

Question

Suppose \(X_{1}, X_{2}, \ldots, X_{n}\) are i.i.d. with $$ X_{i}=\left\\{\begin{aligned} -1 & \text { with probability } 0.2 \\ 1 & \text { with probability } 0.5 \\ 2 & \text { with probability } 0.3 \end{aligned}\right. $$ What can you say about \(\frac{1}{n} \sum_{i=1}^{n} X_{i}\) as \(n \rightarrow \infty\) ?

Step-by-Step Solution

Answer

As \(n\) approaches infinity, \(\frac{1}{n} \sum_{i=1}^{n} X_i\) converges to the expected value, which is 0.9.

Step 1: Calculating Expected Value

Firstly, we calculate the expected value of a single variable, \(X\). This is given by the formula for expectation: \( E[X] = \sum X_i P(X_i) \). So, \( E[X] = (-1)(0.2) + (1)(0.5) + (2)(0.3) \). Calculate each term: \( -1 \times 0.2 = -0.2 \), \( 1 \times 0.5 = 0.5 \), and \( 2 \times 0.3 = 0.6 \). Adding these gives \( E[X] = -0.2 + 0.5 + 0.6 = 0.9 \).

Step 2: Understanding the Law of Large Numbers

The Law of Large Numbers states that as the number of trials \(n\) goes to infinity, the sample average \( \frac{1}{n} \sum_{i=1}^{n} X_i \) converges to the expected value \(E[X]\). This means, for large \(n\), \( \frac{1}{n} \sum_{i=1}^{n} X_i \approx E[X] = 0.9\).

Step 3: Conclusion using the Law of Large Numbers

Since \(X_i\) are i.i.d. (identically independently distributed), the sample average \( \frac{1}{n} \sum_{i=1}^{n} X_i \) will converge to the expected value \(0.9\) as \(n\) approaches infinity, according to the Law of Large Numbers.

Key Concepts

Expected Value

The expected value, often denoted as \( E[X] \), is a fundamental concept in probability that gives us the "average" or "mean" value of a random variable if we were to repeat an experiment an infinite number of times. To calculate it, we multiply each possible outcome by its probability and sum the results. This is represented mathematically as:

• \( E[X] = \sum X_i P(X_i) \)

In our example, the random variable \( X_i \) can take on three values: -1, 1, and 2. Each of these values happens with a certain probability: 0.2, 0.5, and 0.3, respectively.
To find the expected value of \( X_i \), we perform the calculation:

• \( E[X] = (-1)(0.2) + (1)(0.5) + (2)(0.3) \)
• \( = -0.2 + 0.5 + 0.6 \)

Thus, the expected value is \( E[X] = 0.9 \).
This result tells us that if we were to repeatedly draw from this distribution, the average result would approach 0.9.

i.i.d. (independently and identically distributed)

The term i.i.d. is essential in probability and statistics. It means that each random variable in a sequence or collection has the same probability distribution as the others and is statistically independent of them.
This simplifies analysis significantly by allowing the properties of one variable to be known over the entire dataset.
Here’s what this means in simpler terms:

• Identically Distributed: Each variable in the collection follows the same probability distribution. That means they have the same expected value, variance, and so on.
• Independently Distributed: The outcome of one variable doesn’t affect the outcome of another. They are independent events.

In our exercise, since the variables \( X_1, X_2, \ldots, X_n \) are i.i.d., it implies they all share the same expected value of 0.9, and none of their outcomes are influenced by the others.
This concept is crucial because it justifies using the Law of Large Numbers to predict that the average \( \frac{1}{n} \sum_{i=1}^{n} X_i \) will converge to \( E[X] \) as \( n \to \infty \).

Convergence

Convergence is a key concept in probability, particularly when discussing the Law of Large Numbers. It refers to the idea that as you increase the number of trials in an experiment, the sample average will get closer and closer to the expected value.
In mathematical terms, the sample mean \( \frac{1}{n} \sum_{i=1}^{n} X_i \) converges to the expected value \( E[X] \) as \( n \) approaches infinity.

• Strong Law of Large Numbers: This states that the sample averages will almost surely converge to the expected value as \( n \to \infty \).
• Weak Law of Large Numbers: This says that for any small positive value, the probability that the sample average differs from the expected value by more than that value converges to zero as \( n \to \infty \).

In practical terms, convergence tells us that our sample mean \( \frac{1}{n} \sum_{i=1}^{n} X_i \) becomes a reliable estimate of \( E[X] \) once we have a sufficiently large number of observations.
In the exercise's context, it asserts that as we include more data points from this sequence of i.i.d. variables, the average will converge to 0.9.