Problem 73 - An Introduction to Mathematical Statistics and Its Applications

Question

Suppose that \(X\) and \(Y\) have a bivariate normal distribution. (a) Prove that \(X+Y\) has a normal distribution when \(X\) and \(Y\) are standard normal random variables. (b) Find \(E(c X+d Y)\) and \(\operatorname{Var}(c X+d Y)\) in terms of \(\mu_{X}\), \(\mu_{Y}, \sigma_{X}, \sigma_{Y}\), and \(\rho(X, Y)\), where \(X\) and \(Y\) are arbitrary normal random variables.

Step-by-Step Solution

Answer

The sum of two standard normal random variables follows a normal distribution with \(μ=0\) and \(σ= \sqrt{2}\). The expectation of a linear combination \(cX+dY\) is \(c\mu_{X} + d\mu_{Y}\) and the variance is \(c^2\sigma_{X}^2 + d^2\sigma_{Y}^2 + 2cd\rho(X, Y)\sigma_{X}\sigma_{Y}\)

Step 1: Proof of Normal Distribution

Using the concept of moment-generating functions (mgf), we can prove that the sum of standard normal random variables follows a normal distribution. The mgf of a standard normal distribution is \(M_{X}(t)=e^{t^2/2}\). If \(X\) and \(Y\) are independent, their joint mgf is \(M_{X+Y}(t)=M_{X}(t)M_{Y}(t) = e^{t^2} = e^{(t^2/2)^2} \). This is the mgf of a normal distribution with \(μ=0\) and \(σ= \sqrt{2}\)

Step 2: Calculating Expectation

The expected value of \(cX+dY\) can be calculated using the linearity of the expectation: \(E[cX + dY] = cE[X] + dE[Y] = c\mu_{X} + d\mu_{Y}\)

Step 3: Computing Variance

The calculation for variance differs a bit as the variance is not linear. Using the properties of the variance and taking into consideration correlation, the variance of \(cX + dY\) can be calculated as follows: \(\operatorname{Var}(cX + dY) = c^2\variance{X} + d^2\variance{Y} + 2cdCov(X, Y) = c^2\sigma_{X}^2 + d^2\sigma_{Y}^2 + 2cd\rho(X, Y)\sigma_{X}\sigma_{Y}\), where \(Cov(X, Y) = \rho(X, Y)\sigma_{X}\sigma_{Y}\) is the covariance of \(X\) and \(Y\).

Key Concepts

Moment-Generating Functions

Moment-generating functions (mgf) are incredibly powerful tools that help statisticians understand the distribution of random variables. They are named for their ability to generate the moments (mean, variance, skewness, etc.) of a probability distribution. In essence, an mgf, when it exists, uniquely characterizes a probability distribution.

To compute an mgf, we take the expected value of the exponential function of the random variable weighted by a parameter, usually denoted by '\t'. For a standard normal random variable with mean zero and variance one, the mgf is given by: \[ M_{X}(t)=E[e^{tX}]=e^{t^2/2} \] When dealing with the sum of independent normal random variables, say \(X\) and \(Y\), we can find the distribution of the sum \(X+Y\) by multiplying their mgfs: \[ M_{X+Y}(t)=M_{X}(t)M_{Y}(t) \] As shown in the solved exercise, if both \(X\) and \(Y\) are standard normal, the resulting mgf reveals that \(X+Y\) also follows a normal distribution. Therefore, capturing the essence of a distribution through its mgf simplifies many complex problems.

Expectation of Random Variables

The expectation of a random variable, often represented as \( E[X] \) and sometimes called the mean, is a fundamental concept in statistics that embodies the average outcome one would anticipate from an infinite number of repetitions of the same random process. For linear combinations of random variables, such as \( cX+dY \), where \( c \) and \( d \) are constants, the expectation operation retains its linearity. This means that: \[ E[cX+dY] = cE[X]+dE[Y] \] Applying this principle, if X and Y are random variables with known expected values, \( \: \mu_{X} \) and \( \: \mu_{Y} \) respectively, the expected value of \( cX+dY \) can be determined directly as: \[ E[cX + dY] = c\: \mu_{X} + d\: \mu_{Y} \] This feature indicates that to find out the average outcome of a scaled and summed pair of random variables, one simply scales and adds their averages, a concept highlighted in the exercise's step-by-step solution.

Variance and Covariance

Variance and covariance are fundamental statistics concepts that measure the scatter and the joint variability of two random variables, respectively. Variance, denoted as \( \operatorname{Var}(X) \), quantifies how much a random variable \( X \) deviates from its mean \( \mu_{X} \).

Covariance, on the other hand, captures how two random variables \( X \) and \( Y \) change together and is given by \( \operatorname{Cov}(X, Y) \). If the variables tend to show similar behavior—the increment or decrease in one variable leads to the same in the other—covariance is positive. If they behave in opposite ways, it's negative. When they're independent, the covariance is zero.

These measures are connected through the expression of variance of the sum of random variables: \[ \operatorname{Var}(cX + dY) = c^2\operatorname{Var}(X) + d^2\operatorname{Var}(Y) + 2cd\operatorname{Cov}(X, Y) \] For normal random variables, when calculating the variance of a linear combination, we need to incorporate their covariance to reflect how they move together. If \( X \) and \( Y \) have a correlation coefficient \( \rho(X, Y) \), the covariance can alternatively be expressed as \( \rho(X, Y)\: \sigma_{X}\sigma_{Y} \), leading to calculating variance for \( cX+dY \) as shown in the provided exercise.