Problem 2 - An Introduction to Mathematical Statistics and Its Applications

Question

Find the moment-generating function for a chi square random variable and use it to show that \(E\left(\chi_{n}^{2}\right)=n\) and \(\operatorname{Var}\left(\chi_{n}^{2}\right)=2 n\).

Step-by-Step Solution

Answer

The moment generating function for a chi-square random variable \(\chi_{n}^{2}\) is \(\left(\frac{1}{\sqrt{1-2t}}\right)^n\) for \(t < 0.5\). Using this function, the expected value of the chi-square variable is \(n\) and the variance is \(2n\).

Step 1: Determine the moment-generating function (MGF) of a chi-square random variable

If \(X_1, X_2, ..., X_n\) are independent standard normal variables, then the chi-square random variable with \(n\) degrees of freedom, denoted by \(\chi_{n}^{2}\), is \(\chi_{n}^{2} = (X_1^2 + X_2^2 + ... + X_n^2)\). The MGF of \(X_i^2\) is given by \(M(t) = \frac{1}{\sqrt{1-2t}}\) for \(t < 0.5\), therefore, the MGF of \(\chi_{n}^{2}\) is the product of the MGFs of \(X_i^2\) i.e. \(M_{\chi_{n}^{2}}(t) = \left(\frac{1}{\sqrt{1-2t}}\right)^n\) for \(t < 0.5\).

Step 2: Calculation of the expected value \(E\left(\chi_{n}^{2}\right)\)

The expected value \(E(X)\) of a random variable \(X\) can be calculated as the derivative of the MGF at \(t = 0\). Therefore, \(E\left(\chi_{n}^{2}\right) = \frac{d}{dt}M_{\chi_{n}^{2}}(t)\Big|_{t=0} = \frac{n}{2\sqrt{1-2t}}\Big|_{t=0} = n\). This proves the first half of the second part of the exercise.

Step 3: Calculation of the variance \(\operatorname{Var}\left(\chi_{n}^{2}\right)\)

Variance \(Var(X)\) of a random variable \(X\) can be computed as the the second derivative of the MGF at \(t = 0\). After applying the variance formula, \(\operatorname{Var}\left(\chi_{n}^{2}\right) = \frac{d^2}{dt^2}M_{\chi_{n}^{2}}(t)\Big|_{t=0} - [E\left(\chi_{n}^{2}\right)]^2 = \frac{2n}{(1-2t)^2}\Big|_{t=0} - n^2 = 2n\). This last step completes the exercise.

Key Concepts

Chi-square distribution

The chi-square distribution is a fundamental concept in statistics, especially when working with hypotheses tests and confidence intervals. It's a distribution of a sum of the squares of independent standard normal variables. Consider independent random variables:

• Each follows a standard normal distribution.
• You square each variable and then sum them.

This sum, \(X_1^2 + X_2^2 + \ldots + X_n^2\), creates a chi-square distribution with \(n\) degrees of freedom, denoted as \(\chi^2_n\). The degrees of freedom relate to the number of independent variables you are summing.
Chi-square distributions are positively skewed, especially with fewer degrees of freedom, but they become more symmetric as \(n\) increases. This distribution is widely used in tests like the chi-square test for independence and goodness of fit.

Expected value

The expected value, or mean, of a random variable provides a measure of the central tendency. It's similar to the average value you would expect over numerous trials. For the chi-square distribution with \(n\) degrees of freedom, the expected value is straightforward:

• \(E(\chi^2_n) = n\)

This arises due to the additive nature of variance and independence of the normal variables they are derived from. The expected value is useful in summarizing the central behavior of the distribution.
Finding the expected value involves using the moment-generating function (MGF) because it simplifies many calculations. By taking the derivative of the MGF for \(\chi^2_n\) and evaluating it at \(t = 0\), you find the mean: \(n\). This step is crucial in understanding the spread and center of the distribution.

Variance

Variance measures the dispersion or spread in a set of random variables. For a chi-square distribution, variance is particularly insightful as it provides a sense of how much variability there is from the mean.
The variance for \(\chi^2_n\) is:

• \(\operatorname{Var}(\chi^2_n) = 2n\)

Calculating the variance typically requires the use of the second derivative of the moment-generating function at \(t = 0\). It combines with the expected value to highlight distribution characteristics.
Variance calculation involves understanding both the concepts of MGF and algebraic manipulation. Since it's derived from adding squared standard normal variables, the variance scales linearly with degrees of freedom \(n\). Thus, as \(n\) increases, the variability also increases, doubling at each step.