Problem 44 - An Introduction to Mathematical Statistics and Its Applications

Question

A fair coin is tossed two hundred times. Let \(X_{i}=1\) if the \(i\) th toss comes up heads and \(X_{i}=0\) otherwise, \(i=1,2, \ldots, 200 ; X=\sum_{i=1}^{200} X_{i}\). Calculate the Central Limit Theorem approximation for \(P(|X-E(X)| \leq 5)\). How does this differ from the DeMoivre-Laplace approximation?

Step-by-Step Solution

Answer

The σ in this exercise became 0. So, it makes no sense to use either Central Limit Theorem or DeMoivre-Laplace approximations since they are both based on normal distribution approximation and a standard deviation of 0 means it's not normal distribution but a point mass. The expected number of results is the actual results for every experiment.

Step 1: Calculate the Expected Value of \(X\)

Since we're dealing with a fair coin, the chance of getting a head (\(X_{i}=1\)) is \(0.5\). The expected value \(E(X_{i})\) for each toss is then \(1*0.5 + 0*0.5 = 0.5\). As \(X\) is the sum of 200 of these variables (\(X_{i}\)), \(E(X) = 200*E(X_{i}) = 200*0.5 = 100\).

Step 2: Calculate the Standard Deviation of \(X\)

The variance of each \(X_{i}\) is \(Var(X_{i}) = E[(X_{i})^2] - [E(X_{i})]^2\). Since \(X_{i}\) is a Bernoulli variable, \(E[(X_{i})^2] = [E(X_{i})]^2 = 0.5^2 = 0.25\). So, \(Var(X_{i}) = 0.25 - 0.25 = 0. The standard deviation \(σ\) of \(X\) is the square root of the sum of 200 of these variances, So, \(σ = \sqrt{200*0} = 0\)

Step 3: Find the Central Limit Theorem Approximation

Next, we normalize \(X\), which can be written as \(Z = (X - E(X)) / σ\). The Central Limit Theorem tells us that the sum \(X\) of a large number of binary random variables is approximately normally distributed. So, we are looking to approximate \(P(|X - E(X)| ≤ 5)\) which now becomes \(P(|Z| ≤ 5/σ)\). However, because the price of \(σ\) is \(0\), it makes no sense to divide by \(σ\). The Central Limit Theorem does not apply in this case, because \(σ = 0\).

Step 4: Considerations about the DeMoivre-Laplace Approximation

DeMoivre-Laplace approximation is usable when estimating the distribution of a sum of a large number of binary random variables. It is similar to the Central Limit Theorem approximation. In fact, the most differences between them are due to the nature of approximation involved, not due to fundamental differences between the theorems. However, in this case, as our standard deviation \(σ\) became \(0\), using the DeMoivre-Laplace approximation will lead to a similar nonsensical result as when we use Central Limit Theorem.

Key Concepts

Bernoulli distribution

In probability theory, a Bernoulli distribution is a simple yet fundamental concept. It models a binary outcome scenario, meaning that an event can only result in two possible outcomes: success or failure. An excellent example is a single coin flip, where getting a "head" is a success, and a "tail" is a failure.

The Bernoulli distribution is described by a parameter known as the probability of success, denoted as \( p \). If \( p = 0.5 \), as with a fair coin, both outcomes are equally likely. This distribution is crucial as it paves the way to understanding more complex distributions, like the Binomial distribution, which is a sequence of Bernoulli trials.

• Probability of Success (\( X = 1 \)): \( P(X = 1) = p \)
• Probability of Failure (\( X = 0 \)): \( P(X = 0) = 1 - p \)

Expected value

Expected value is a key concept in statistics and probability that allows us to forecast the average outcome of a random variable over a large number of trials. For a single random variable \( X \) following a Bernoulli distribution, the expected value is calculated based on the probabilities of the outcomes.

For a Bernoulli variable \( X_i \), when the probability of success is \( 0.5 \), the expected value \( E(X_i) \) is \( 0.5 \).

• Formula for Expected Value: \( E(X) = 1 \times p + 0 \times (1-p) \)
• For a fair coin toss, expected value is simply \( 0.5 \).

In this exercise, with 200 tosses, the expected value of the sum of all these variables (denoted by \( X \)) becomes 100, as calculated by multiplying 200 by the individual expected value, \( E(X) = 200 \times 0.5 \). This prediction means, on average, we expect 100 heads in 200 tosses.

Variance and standard deviation

Variance and standard deviation are closely linked concepts that measure variability or spread in a dataset. Variance gives us an idea of how much each random variable differs from the expected value statistically.

In this example, where each trial is a Bernoulli random variable, variance is computed using the squared difference from the expected value.

• Variance Formula for Bernoulli Variable: \( Var(X_i) = E[(X_i)^2] - [E(X_i)]^2 \)
• Since \( E(X_i) = 0.5 \) for a fair coin, \( E[(X_i)^2] = (0.5)^2 = 0.25 \), thus variance is \( 0.25 - 0.25 = 0 \).
• Standard deviation, denoted by \( \sigma \), is the square root of variance. Thus, for the Bernoulli variable, \( \sigma = \sqrt{0} = 0 \).

In our exercise, the problem arises from the standard deviation being zero, making it inapplicable for further Central Limit Theorem calculations.

DeMoivre-Laplace approximation

The DeMoivre-Laplace approximation is a method used to approximate the distribution of a binomial condition with a normal distribution. It closely parallels the Central Limit Theorem in its application to large sets of binary random variables.

Where the binomial distribution has large numbers of trials \( n \), the DeMoivre-Laplace theorem allows us to treat the distribution like a normal one.

• This approximation operates well under conditions where \( n \) is large, making complex calculations simpler by using the normal distribution's characteristics.
• In normal circumstances, the standard deviation \( \sigma \) of the sum is necessary to normalize the sum for this approximation.

In this exercise, the zero variance and standard deviation halt the practical application of both the Central Limit Theorem and the DeMoivre-Laplace approximation, rendering them both unusable in this scenario.