Q.7.20 - A First Course in Probability

Question

The Conditional Covariance Formula. The conditional covariance of $X$and $Y$, given $Z$ is defined by $Cov(X,Y\mid Z)\equiv E\left[\right(X-E[X\mid Z]\left)\right(Y-E[Y\mid Z])\mid Z]$

a) Show that $Cov(X,Y\mid Z)=E[XY\mid Z]-E[X\mid Z]E[Y\mid Z]$

b) Prove the conditional covariance formula $\mathrm{Cov}(X,Y)=E\left[\mathrm{Cov}\right(X,Y\mid Z\left)\right]+\mathrm{Cov}\left(E\right[X\mid Z],E[Y\mid Z\left]\right)$

c) Set $X=Y$in part (b) and obtain the conditional variance formula.

Step-by-Step Solution

Answer

a) It has been shown that $\mathrm{Cov}(X,Y\mid Z)=E[XY\mid Z]-E[X\mid Z]E[Y\mid Z]$

b) The conditional covariance formula has been proved 

$\mathrm{Cov}(X,Y\mid Z)=E[XY\mid Z]-E[X\mid Z]E[Y\mid Z]$

c) The conditional variance formula is $\mathrm{Var}\left(\mathrm{Y}\right)=\mathrm{E}\left[\mathrm{Var}\right(\mathrm{Y})\mid \mathrm{Z}]+\mathrm{Var}\left[\mathrm{E}\right(\mathrm{Y}\mid \mathrm{Z}\left)\right]$

Step 1: Given Information (Part a)

Show that $Cov(X,Y\mid Z)\equiv E\left[\right(X-E[X\mid Z]\left)\right(Y-E[Y\mid Z])\mid Z]$

Step 2: Explanation (Part a)

We are given that,

$\begin{array}{r}\mathrm{Cov}(X,Y\mid Z)=E[X-E(X\mid Z\left)\right(Y-E(Y\mid Z)\mid Z\left)\right]\end{array}$

$\begin{array}{r}\Rightarrow \mathrm{Cov}(X,Y\mid Z)=E[XY-XE(Y\mid Z)-YE(X\mid Z)+E(X\mid Z)\cdot E(Y\mid Z)\mid Z]\end{array}$

$=E(XY\mid Z)-E\left[XE\right(Y\mid Z)\mid Z]-E\left[YE\right(X\mid Z)\mid Z]+E\left[E\right(X\mid Z\left)E\right(Y\mid Z)\mid Z]$

$=E[XY\mid Z]-E(X\mid Z)·E(Y\mid Z)-E(Y\mid Z)·E(X\mid Z)+E[X\mid Z]·E[Y\mid Z]$

$=E[XY\mid Z]-E[X\mid Z]·E[Y\mid Z]$

Step 3: Final Answer (Part a)

It has been shown that $\mathrm{Cov}(X,Y\mid Z)=E[XY\mid Z]-E[X\mid Z]E[Y\mid Z].$

Step 1: Given Information (Part b)

The conditional covariance formula $=\mathrm{Cov}(X,Y)=E\left[\mathrm{Cov}\right(X,Y\mid Z\left)\right]+\mathrm{Cov}\left(E\right[X\mid Z],E[Y\mid Z\left]\right)$

Step 2: Explanation (Part b)

b) $\text{ R.H.S. }=E\left[\mathrm{Cov}\right(X,Y\mid Z\left)\right]+\mathrm{Cov}\left[E\right(X\mid Z),E(Y\mid Z\left)\right]$

Using Result of part (a) 

$\begin{array}{r}=E\left[E\right(XY\mid Z)-E(X\mid Z\left)E\right(Y\mid Z\left)\right]+E\left[E\right(X\mid Z)\cdot E(Y\mid Z)\mid Z]-E\left[E\right(X\mid Z)\mid Z]\cdot E\left[E\right(Y\mid Z)\mid \end{array}$

$\begin{array}{r}=E\left(XY\right)-E\left(X\right)E\left(Y\right)+E\left(X\right)\cdot E\left(Y\right)-E\left(X\right)\cdot E\left(Y\right)\end{array}$

$=E\left(XY\right)-E\left(X\right)\cdot E\left(Y\right)$

$=\mathrm{Cov}(X,Y)$

Step 3: Final Answer (part b)

Therefore, the conditional covariance formula $\mathrm{Cov}(X,Y)=E\left[\mathrm{Cov}\right(X,Y\mid Z\left)\right]+\mathrm{Cov}\left(E\right[X\mid Z],E[Y\mid Z\left]\right)$has been proved. 

Step 1: Given Information (Part c)

Set $X=Y$ in part (b)

Step 2: Explanation (Part c)

c) Putting $X=Y$ in result of part (b)

$\mathrm{Cov}(X,X)=E\left[\mathrm{Cov}\right(X,X)\mid Z]+\mathrm{Cov}\left[E\right(X\mid Z),E(X\mid Z\left)\right]$

$\Rightarrow \mathrm{Var}\left(X\right)=E\left[\mathrm{Var}\right(X)\mid Z]+\mathrm{Var}\left[E\right(X\mid Z\left)\right]$

Similarly $\mathrm{Var}\left(\mathrm{Y}\right)=\mathrm{E}\left[\mathrm{Var}\right(\mathrm{Y})\mid \mathrm{Z}]+\mathrm{Var}\left[\mathrm{E}\right(\mathrm{Y}\mid \mathrm{Z}\left)\right]$

Step 3: Final Answer

Therefore, the conditional variance formula is $\mathrm{Var}\left(Y\right)=E\left[\mathrm{Var}\right(Y)\mid Z]+\mathrm{Var}\left[E\right(Y\mid Z\left)\right]$.