Problem 72 - An Introduction to Mathematical Statistics and Its Applications

Question

Suppose that \(X\) and \(Y\) have a bivariate normal distribution with \(\operatorname{Var}(X)=\operatorname{Var}(Y)\). (a) Show that \(X\) and \(Y-\rho X\) are independent. (b) Show that \(X+Y\) and \(X-Y\) are independent. [Hint: See Question 11.4.7(a).]

Step-by-Step Solution

Answer

By calculating the covariance and proving it to be zero, it is shown that in both cases, the pairs \(X\) and \(Y-\(\rho\)X, and \(X+Y\) and \(X-Y\) are independent.

Step 1: Analyzing the 1st Part of the Problem

To show that X and Y-\(\rho\)X are independent, one can accomplish this by verifying that the covariance of X and Y-\(\rho\)X equals zero, because in a bivariate normal distribution, zero covariance implies independence.

Step 2: Calculating covariance for X and Y-\(\rho\)X

The covariance is given by \[\Cov(X,Y-\rho X) = E[(X-E[X])(Y-\rho X-E[Y-\rho X])] = E[XY-\rho X^2] - E[X]E[Y-\rho X]\]As the givens are \Var(X)=\Var(Y), so E[X^2]= \Var(X)= \Var(Y) and E[X]=E[Y]=0 because any variable subtracted by itself equals 0.Thus, \[\Cov(X,Y-\rho X) = E[XY-\rho X^2] \]Inserting the given, that \(\Var(X)=\Var(Y)\), we see that the covariance equals to zero. Therefore, \(X\) and \(Y-\rho X\) are independent.

Step 3: Analyzing The 2nd Part of the Problem

Next, we need to show that \(X+Y\) and \(X-Y\) are independent. Similar to the first case, this can be accomplished by showing that their covariance is zero.

Step 4: Calculating Covariance for X+Y and X-Y

The covariance is given by \[\Cov(X+Y, X-Y) = E[((X+Y)- E[X+Y])((X-Y)- E[X-Y])] = E[X^2 - Y^2] - E[X+Y]E[X-Y]Again, as the givens are \Var(X)=\Var(Y), so E[X^2] = \Var(X) = \Var(Y) and using E[X]=E[Y]=0 because any variable subtracted by itself equals 0.Thus,\[\Cov(X+Y, X-Y)= E[X^2 - Y^2]\]Since \Var(X)=\Var(Y), hence we see that the covariance equals to zero.Therefore, \(X+Y\) and \(X-Y\) are independent.

Key Concepts

Covariance

Covariance is a statistical measure that indicates the extent to which two variables change together. In the context of a bivariate normal distribution, it is crucial to understand if and how variables are related. When the covariance of two variables is zero, this can often imply independence, especially in normally distributed variables like in the exercise above. To compute covariance between two variables, say \(X\) and \(Y\), we use the formula:

• \( \Cov(X, Y) = E[(X - E[X])(Y - E[Y])] \)

This formula captures the joint variability of the variables around their means.
When applied in scenarios where bivariate normal distributions are involved, like in the given exercise, zero covariance helps us conclude that the transformed variable pairs \(X\) and \(Y - \rho X\), and \(X + Y\) and \(X - Y\) become independent.

Independence

Independence in statistics means that the occurrence of one event does not affect the probability of another. In particular, for two random variables to be independent, their joint probability distribution must be the product of their individual probability distributions. In the context of a bivariate normal distribution, independence can often be identified through the covariance. As we saw in the exercise:

• If \( \Cov(X, Y - \rho X) = 0 \), \(X\) is independent of \(Y - \rho X\).
• If \( \Cov(X + Y, X - Y) = 0 \), \(X + Y\) is independent of \(X - Y\).

Zero covariance is a strong indicator of independence when dealing with linear combinations of normally distributed variables. This property simplifies the analysis of complex relationships between variables.

Variance

Variance measures the spread of a set of numbers. Specifically, for a random variable, it quantifies how much the values of the variable deviate from the mean. It is calculated as:

• \( \Var(X) = E[(X - E[X])^2] \)

In our exercise, we assume \( \Var(X) = \Var(Y) \), which simplifies calculations significantly.
This property helps us to equate certain expressions, like \( E[X^2] = \Var(X) \), leading to the realization that when \( \Var(X) = \Var(Y) \), certain combinations like \(X + Y\) and \(X - Y\) have zero covariance, indicating independence.
Understanding variance in this context is key to solving and simplifying problems involving normally distributed variables with equal variances.

Expectation

Expectation, often denoted \(E[X]\), is the mean or average value that a random variable takes. It is foundational in defining other statistical measures like variance and covariance. In simpler terms, expectation is what you "expect" the value of a random variable to be — its central tendency. For a continuous random variable \(X\), the expectation can be calculated using:

• \( E[X] = \int x f_X(x) \, dx \)

where \(f_X(x)\) is the probability density function of \(X\).
In our bivariate normal distribution exercise, by knowing \(E[X] = E[Y] = 0\), it simplifies the covariance calculations considerably.
Such assumptions help demonstrate the independence of variable pairs as their zero mean simplifies other complex expressions involved in the covariance calculations.