Problem 30 - An Introduction to Mathematical Statistics and Its Applications

Question

Let \(X_{1}, X_{2}, \ldots, X_{n}\) and \(Y_{1}, Y_{2}, \ldots, Y_{m}\) be independent random samples from normal distributions with means \(\mu_{X}\) and \(\mu_{Y}\) and standard deviations \(\sigma_{X}\) and \(\sigma_{Y}\), respectively, where \(\mu_{X}\) and \(\mu_{Y}\) are known. Derive the GLRT for \(H_{0}: \sigma_{X}^{2}=\sigma_{Y}^{2}\) versus \(H_{1}: \sigma_{X}^{2}>\sigma_{Y}^{2}\).

Step-by-Step Solution

Answer

For the GLRT of \(H_{0}: \sigma_{X}^{2} = \sigma_{Y}^{2}\) versus \(H_{1}: \sigma_{X}^{2} > \sigma_{Y}^{2}\), the likelihood ratio test statistic is \(\lambda = (S_{Y}^{2}/S_{X}^{2})^{m/2} (n/m)^{n/2}\) where \(S_{X}^{2}\) and \(S_{Y}^{2}\) are the sample variances of the two samples.

Step 1: State the Null and Alternative Hypotheses

Let's start with stating the null hypothesis and the alternative hypothesis. The null hypothesis is \(H_{0}: \sigma_{X}^{2} = \sigma_{Y}^{2}\). This means that the variances of the two distributions are equal. The alternative hypothesis is \(H_{1}: \sigma_{X}^{2} > \sigma_{Y}^{2}\). This implies that the variance of the first population is greater than the variance of the second population.

Step 2: Derive the likelihood function

Given that the samples are from normal distributions, the likelihood functions for \(X\) and \(Y\) are \[L(X | \sigma_{X}^{2}, \mu_{X}) = \prod_{i=1}^{n} \frac{1}{\sqrt{2\pi \sigma_{X}^{2}}}e^{-\frac{(X_{i}-\mu_{X})^{2}}{2\sigma_{X}^{2}}}\] and \[L(Y | \sigma_{Y}^{2}, \mu_{Y}) = \prod_{j=1}^{m} \frac{1}{\sqrt{2\pi \sigma_{Y}^{2}}}e^{-\frac{(Y_{j}-\mu_{Y})^{2}}{2\sigma_{Y}^{2}}}\] respectively.

Step 3: Compute the Likelihood Ratio

The likelihood ratio test statistic is defined as \[λ = \frac{sup_{H_{0}} L(X, Y; \sigma_{X}^{2}, \sigma_{Y}^{2} )}{sup_{H_{1}} L(X, Y; \sigma_{X}^{2}, \sigma_{Y}^{2} )}\] where \(sup_{H_{0}}\) and \(sup_{H_{1}}\) are the supremum of the likelihood function over the parameter space under the null and alternative hypotheses respectively. This will need several computations by substituting the hypotheses into the equations. Also, we need to take the logarithm to simplify the calculations and then maximize the test statistic.

Step 4: Finalize the Test Statistic

The supremum of the likelihood ratio test statistic under the null hypothesis will be at \(\sigma_{X}^{2} = \sigma_{Y}^{2}\) equals some value \(\sigma^{2}\) and the supremum under the alternative hypothesis will be at \(\sigma_{X}^{2} = S_{X}^{2}\) and \(\sigma_{Y}^{2} = S_{Y}^{2}\). After processing the equations, the likelihood ratio test statistic simplifies to \(\lambda = (S_{Y}^{2}/S_{X}^{2})^{m/2} (n/m)^{n/2}\) where \(S_{X}^{2}\) and \(S_{Y}^{2}\) are the sample variances of the two samples.

Key Concepts

Hypothesis Testing

Hypothesis testing is pivotal in statistics, allowing us to make inferences about population parameters based on sample data. It involves two main hypotheses: the null hypothesis (\(H_{0}\)) and the alternative hypothesis (\(H_{1}\)).
In our scenario:

• The null hypothesis, \(H_{0}: \sigma_{X}^{2} = \sigma_{Y}^{2}\), assumes the variances of the two populations are equal.
• The alternative hypothesis, \(H_{1}: \sigma_{X}^{2} > \sigma_{Y}^{2}\), suggests that the variance of the first population is greater than that of the second.

A hypothesis test evaluates these hypotheses using sample data, often leveraging statistical tests to determine how likely the observed data would occur if the null hypothesis were true. We then decide to accept or reject \(H_{0}\). A common threshold for decision-making is the significance level, often set at 0.05.

Normal Distribution

Understanding the normal distribution is crucial, especially since our samples come from normal distributions.
The normal distribution:

• Is a continuous probability distribution that is symmetrical around its mean, \(\mu\).
• Characterizes how data is dispersed in a bell-shaped curve, where most observations cluster around the mean.

For our problem, each sample, \(X\) and \(Y\), comes from a population with a normal distribution but potentially different variances, \(\sigma_{X}^{2}\) and \(\sigma_{Y}^{2}\). Knowing these distributions allows us to derive the likelihood functions for the data samples and apply the Generalized Likelihood Ratio Test effectively.

Variance Comparison

When comparing variances, we often use statistical tests to determine if they differ significantly between two populations.
In our exercise:

• The goal is to test if \(\sigma_{X}^{2}\) is greater than \(\sigma_{Y}^{2}\).
• The Generalized Likelihood Ratio Test helps us evaluate this by forming a ratio of likelihoods under different hypotheses.

The test statistic is computed using the sample variances, \(S_{X}^{2}\) and \(S_{Y}^{2}\), derived from the data.
The simplified ratio, \(\lambda = (S_{Y}^{2}/S_{X}^{2})^{m/2} (n/m)^{n/2}\), captures the relationship between the variances, providing a basis to accept or reject the null hypothesis based on whether this ratio indicates significant differences.