Problem 9 - A Modern Introduction to Probability and Statistics: Understanding Why and How

Question

Let \(X_{1}, X_{2}, \ldots\) be an independent sequence of \(U(-1,1)\) random variables and let \(T_{n}=\frac{1}{n} \sum_{i=1}^{n} X_{i}^{2} .\) It is claimed that for some \(a\) and any \(\varepsilon>0\) $$ \lim _{n \rightarrow \infty} \mathrm{P}\left(\left|T_{n}-a\right|>\varepsilon\right)=0 . $$ a. Explain how this could be true. b. Determine \(a\).

Step-by-Step Solution

Answer

b. The value of \(a\) is \(\frac{1}{3}\).

Step 1: Understand the Claim

The claim suggests that for some value \(a\), the sequence \(T_n\) converges in probability to \(a\) as \(n\) approaches infinity. This implies the law of large numbers might be applicable, suggesting \(T_n\) is a consistent estimator of \(a\).

Step 2: Setup for Covariance and Expectation

Given that \(X_i\) are independent and uniformly distributed over \((-1, 1)\), we evaluate the properties of \(X_i^2\). Start by calculating \(E[X_i^2]\), because this will help in finding \(E[T_n]\). If \(T_n\) converges to \(a\) in probability, it should theoretically also converge to \(E[X_i^2]\), provided the assumptions like existence of finite moments are met.

Step 3: Calculate Expected Value of Each Square

To find \(E[X_i^2]\), integrate over the distribution of \(X_i\). Since \(X_i\) is uniform over \((-1, 1)\), we have: \[ E[X_i^2] = \int_{-1}^{1} x^2 \cdot \frac{1}{2} \,dx = \frac{1}{2} \int_{-1}^{1} x^2 \,dx. \] Evaluate this to find the result.

Step 4: Evaluation of the Integral

Solving the integral: \[ \int_{-1}^{1} x^2 \,dx = \left[ \frac{x^3}{3} \right]_{-1}^{1} = \left( \frac{1}{3} \right) - \left( -\frac{1}{3} \right) = \frac{2}{3}. \] Subsequently, \( E[X_i^2] = \frac{1}{2} \times \frac{2}{3} = \frac{1}{3}. \)

Step 5: Apply Law of Large Numbers

According to the law of large numbers, \(T_n = \frac{1}{n} \sum_{i=1}^{n} X_i^2\) will converge to \(E[X_i^2] = \frac{1}{3}\) as \(n\) approaches infinity. Thus, by the strong law, for all \(\varepsilon > 0\), \(\lim_{n \to \infty} P(|T_n - \frac{1}{3}| > \varepsilon) = 0\), confirming the claim.

Key Concepts

Uniform Distribution

In statistics, the Uniform Distribution is a type of probability distribution where all outcomes are equally likely within a certain interval. For our exercise, the random variables \(X_i\) are drawn from a uniform distribution over the interval \((-1, 1)\). This simply means that every point within this interval has an equal chance of being selected.

Uniform distributions are popular for their simplicity and ease of calculation. They are defined by just two parameters: the minimum and maximum values of the interval. In this case, those values are \(-1\) and \(1\).

One characteristic of the uniform distribution is that the mean, or expected value, can be calculated as the midpoint of the interval. However, in this problem, we are interested in the expected value of \(X_i^2\). Since \(X_i\) is uniform, its squared value \(X_i^2\) will weigh probabilities towards zero, due to squaring. This is why (as shown in the solution steps) you need to integrate to find the expectation of \(X_i^2\), not just average the endpoints. These calculations give insight into how the uniform distribution impacts problem-solving through expectation.

Convergence in Probability

Convergence in probability is a fundamental concept in probability theory. It essentially looks at how a sequence of random variables becomes less and less random as the sample size grows.

In our problem, we have the sequence \(T_n = \frac{1}{n} \sum_{i=1}^{n} X_i^2\). The problem statement and solution indicate that this sequence converges in probability to a specific value, \(a\), as \(n\) approaches infinity. What this means in simple terms is that as you take more and more samples (or trials), the average value calculated from them gets closer and closer to this true value, and the probability that the value is significantly different from \(a\) (by any positive \(\varepsilon\)) decreases to zero.

To understand why this happens, consider that each sample \(X_i^2\) has some randomness, but when you average enough of them together as shown by \(T_n\), this randomness averages out. The key backing concept here is the Law of Large Numbers, which tells us that as \(n\) increases, the sample average will converge to the expected value \(E[X_i^2]\), reducing variability. Thus, this convergence assures us that for a large number of trials, the average should be close to the expected value.

Expectation and Variance

Expectation and variance are two crucial statistical concepts used to describe distributions of random variables.

**Expectation**
The expected value, or mean, of a random variable gives us a measure of the central tendency - essentially, where values cluster around on average. In this exercise, we calculated the expected value of \(X_i^2\), where \(X_i\) follows a uniform distribution on \((-1, 1)\). Through integration, we discovered this value to be \(\frac{1}{3}\). This number symbolizes the value that individual \(X_i^2\) values average out to over many samples.

**Variance**
Variance, on the other hand, measures the spread or dispersion of a set of values. It tells us how much individual values tend to deviate from the expected value. While variance wasn't directly calculated in the given solution, it's important to recognize that a small variance contributes to the convergence in probability. This is because smaller variance means less deviation, leading to the sample means being more tightly clustered around the true mean with larger samples.

Understanding both these statistics helps us get a full picture of how the sequence \(T_n\) behaves – not only what typical values we might expect due to expectation but also how they might vary, thanks to variance.