Problem 61 - Applied Statistics and Probability for Engineers

Question

If \(X\) and \(Y\) have a bivariate normal distribution with joint probability density \(f_{X Y}\left(x, y ; \sigma_{X}, \sigma_{Y}, \mu_{X}, \mu_{Y}, \rho\right),\) show that the marginal probability distribution of \(X\) is normal with mean \(\mu_{x}\) and standard deviation \(\sigma_{x} .\) [Hint: Complete the square in the exponent and use the fact that the integral of a normal probability density function for a single variable is \(1 .]\)

Step-by-Step Solution

Answer

The marginal distribution of X is normal with mean \(\mu_X\) and standard deviation \(\sigma_X\).

Step 1: Write the bivariate normal PDF

The joint PDF of \((X,Y)\) is \(f_{XY}(x,y) = \frac{1}{2\pi\sigma_X\sigma_Y\sqrt{1-\rho^2}} \exp\left(-\frac{1}{2(1-\rho^2)}\left[\frac{(x-\mu_X)^2}{\sigma_X^2} - \frac{2\rho(x-\mu_X)(y-\mu_Y)}{\sigma_X\sigma_Y} + \frac{(y-\mu_Y)^2}{\sigma_Y^2}\right]\right)\)

Step 2: Integrate over y to find marginal of X

\(f_X(x) = \int_{-\infty}^{\infty} f_{XY}(x,y)\, dy\). Substitute \(u = \frac{y-\mu_Y}{\sigma_Y}\) and complete the square in \(u\) in the exponent. The exponent becomes \(-\frac{(x-\mu_X)^2}{2\sigma_X^2} - \frac{1}{2(1-\rho^2)}\left(u - \rho\frac{x-\mu_X}{\sigma_X}\right)^2\).

Step 3: Evaluate the Gaussian integral

The integral over \(u\) of \(\exp\left(-\frac{(u-c)^2}{2(1-\rho^2)}\right)\) gives \(\sqrt{2\pi(1-\rho^2)}\). After combining all constants:
\(f_X(x) = \frac{1}{\sqrt{2\pi}\sigma_X}\exp\left(-\frac{(x-\mu_X)^2}{2\sigma_X^2}\right)\)

Step 4: Conclusion

This is the PDF of a normal distribution with mean \(\mu_X\) and standard deviation \(\sigma_X\). Therefore, the marginal distribution of \(X\) is \(N(\mu_X, \sigma_X^2)\).

Key Concepts

Marginal Probability Distribution

A marginal probability distribution helps us understand the probability of one single variable within a multivariate distribution. When dealing with two random variables that form a bivariate normal distribution, we can focus on one variable by integrating out the other. This process gives us the marginal probability distribution of the variable of interest.

For instance, if we have random variables \(X\) and \(Y\) with a joint distribution, determining the marginal distribution of \(X\) involves integrating the joint probability density function with respect to \(Y\). This results in the marginal distribution of \(X\), which turns out to be normally distributed. The mean \(\mu_X\) and standard deviation \(\sigma_X\) are the same as those specified in the joint distribution.

This concept is fundamental because it allows us to study multivariate distributions by simplifying them to one dimension. Whether for statistical analysis or practical applications, understanding the behavior of each variable independently can provide significant insights.

Joint Probability Density Function

The joint probability density function (PDF) is crucial in describing the likelihood of two random variables occurring simultaneously. In the bivariate normal distribution, the joint PDF shows us how two random variables, \(X\) and \(Y\), are distributed together. The joint PDF includes parameters like the means \(\mu_X, \mu_Y\), standard deviations \(\sigma_X, \sigma_Y\), and the correlation \(\rho\) between the two variables.

This function is more complex than a univariate normal distribution since it needs to account for the interaction between \(X\) and \(Y\). It includes terms that express the variation of each variable and the interaction term that involves \(\rho\). The joint PDF is given by a formula:

• Normalization constant: \(\frac{1}{2\pi \sigma_X \sigma_Y \sqrt{1 - \rho^2}}\)
• Exponent: \(-\frac{1}{2(1 - \rho^2)} \left[ \frac{(x-\mu_X)^2}{\sigma_X^2} + \frac{(y-\mu_Y)^2}{\sigma_Y^2} - \frac{2\rho(x-\mu_X)(y-\mu_Y)}{\sigma_X \sigma_Y} \right]\)

The joint PDF provides a full picture of how both variables behave and interact, which is essential for understanding their combined effect.

Completing the Square

Completing the square is a mathematical technique used to simplify expressions, especially quadratics. In the context of the bivariate normal distribution, we use this method to isolate and simplify terms to make them more convenient for integration.

In the given exercise, we deal with a complex exponent in the joint probability density function. By completing the square in its exponent, we can rearrange the terms to look like separate squares, making integration more feasible.

Here's a brief on how completing the square works:

• Take the quadratic form in the exponent: \((x - \mu_X)^2\) and \((y - \mu_Y)^2\).
• Adjust it by incorporating the cross term \(- \frac{2\rho(x-\mu_X)(y-\mu_Y)}{\sigma_X \sigma_Y}\).
• Rearrange into a sum of squared terms.

This allows us to integrate out the variable \(Y\) easily, focusing on the variable \(X\) to find its marginal distribution. By doing this, we see how integration simplifies when terms are neatly squared and set up for standard integration techniques known from single-variable calculus.