Problem 50 - A First Course in Probability

Question

Let \(Z_{1}\) and \(Z_{2}\) be independent standard normal random variables. Show that \(X, Y\) has a bivariate normal distribution when \(X=Z_{1}, Y=Z_{1}+Z_{2}.\)

Step-by-Step Solution

Answer

To show that the given \(X = Z_1\) and \(Y = Z_1 + Z_2\) follow a bivariate normal distribution, we considered the random variable \(W = aX + bY\), where \(a\) and \(b\) are real numbers. Then, we substituted the definitions of \(X\) and \(Y\) and simplified to get \(W = (a + b)Z_1 + bZ_2\). Since \(Z_1\) and \(Z_2\) are independent standard normal random variables and normal variables are closed under linear combinations, it follows that \(W\) is normally distributed. Therefore, the random vector \((X, Y)\) has a bivariate normal distribution.

Step 1: Recall the definition of a bivariate normal distribution

A random vector \((X, Y)\) follows a bivariate normal distribution if and only if every linear combination of \(X\) and \(Y\) is normally distributed. That is, for any real numbers \(a\) and \(b\), the random variable \(aX + bY\) has a normal distribution.

Step 1: Define arbitrary linear combinations

Let's consider the random variable \(W = aX + bY\), where \(a\) and \(b\) are real numbers. By substituting the definitions of \(X\) and \(Y\), we get \(W = aZ_1 + b(Z_1 + Z_2)\).

Step 2: Simplify the linear combination

Now, we can simplify the random variable \(W\) by collecting terms with \(Z_1\) and \(Z_2\) as shown below: \(W = (a + b)Z_1 + bZ_2\)

Step 3: Recognize the normal distribution

Since \(Z_1\) and \(Z_2\) are independent standard normal random variables and \(a + b\) and \(b\) are constants, it is clear that the random variable \(W\) is also normally distributed. This is because the sum of independent normal variables is also a normal variable.

Step 4: Conclude that the distribution is bivariate normal

As we have shown that for any real numbers \(a\) and \(b\), the linear combination \(W = aX + bY\) is normally distributed, it follows that the random vector \((X, Y)\) has a bivariate normal distribution.

Key Concepts

Standard Normal Random Variables

When we talk about standard normal random variables, we are referring to a special category of random variables that have a particular type of distribution known as the normal, or Gaussian, distribution. This distribution is symmetric around the mean, which is zero, and it has a standard deviation (and variance) of one.

This is incredibly important in the field of statistics because the properties of the standard normal distribution are well understood and can provide us with a lot of information about probabilities associated with these variables. For a random variable to be considered standard normal, it must be transformed such that it aligns with these properties. This is often done through a process called standardization, where we subtract the mean and divide by the standard deviation of the original variable.

For example, if we take any normal random variable with mean \textmu and standard deviation \textsigma, and apply the transformation \[ Z = \frac{(X - \textmu)}{\textsigma} \],we get a standard normal random variable denoted as Z. Such variables are integral to the bivariate normal distribution when we deal with two of them, as in the original exercise.

Linear Combination of Random Variables

The term linear combination of random variables refers to an expression involving random variables that are each multiplied by a constant and then summed. For example, if you have two random variables, X and Y, and two constants, a and b, a linear combination would be \[ W = aX + bY \].

In probability and statistics, linear combinations are particularly interesting when we think about how they distribute. The exercise provided makes use of this concept by examining the distribution of W, a linear combination of standard normal random variables X and Y. One of the fascinating properties of the normal distribution is that the sum (or linear combination) of independent normal random variables is itself normally distributed.

Understanding this property is crucial, as it allows us to make predictions and calculate probabilities for a wide range of outcomes that depend on several random variables working in tandem.

Independence in Probability

In the realm of probabilities, independence is a cornerstone concept that refers to two events not affecting each other's outcomes. When dealing with random variables, like in our exercise's case of Z1 and Z2, independence implies that knowing the value of one does not provide any information about the value of the other. This is a critical assumption when working with multiple random variables, and it greatly simplifies the analysis of their joint behavior.

For instance, if Z1 and Z2 are independent, the probability of Z1 falling within a certain range remains unchanged regardless of Z2's value, and vice versa. In the analysis of the bivariate normal distribution, this property of independence is vital. It allows us to assert that the sum or difference of the variables—as long as the variables are normally distributed and independent—will also follow a normal distribution. Consequently, the bivariate normal distribution's attributes, where X and Y are defined as a combination of Z1 and Z2, heavily rely on this independence.