Q.7.42 - A First Course in Probability

Question

It follows from Proposition $6.1$ and the fact that the best linear predictor of $Y$ with respect to $X$ is $\mu y + \rho \sigma y/\sigma x (X - \mu x)$ that if $E\left[Y\right|X] = a + bX$ then $a = \mu y - \rho \sigma y/\sigma x \mu x$ $b = \rho \sigma y \sigma x$ (Why?) Verify this directly 

Step-by-Step Solution

Answer

Minimize the expected squared error.

Step 1: Given Information

$a={\mu }_y-\rho \frac{{\sigma }_y}{{\sigma }_x}{\mu }_x$ , $b=\rho \frac{{\sigma }_y}{{\sigma }_x}$

Step 2: Explanation

Let's find constants $a$ and $b$ such that minimizes the expected squared error

$E\left((Y-a-bX{)}^2\right)$

If we differentiate it partially respective to $a$, we end up with condition  $a+b E\left(X\right)=E\left(Y\right)$

and if we differentiate it partially respective to $b$, we end up with condition

$aE\left(X\right)+bE\left(X^2\right)=E\left(XY\right)$

Step 3: Explanation

The system of equations,

$\left\{\begin{array}{lll}a& +bE\left(X\right)& =E\left(Y\right)\\ aE\left(X\right)& +bE\left(X^2\right)& =E\left(XY\right)\end{array}\right.$

and if we solve it, we get

$b=\frac{Cov(X,Y)}{Var\left(X\right)}$

$=\rho ·\frac{{\sigma }_y}{{\sigma }_x}$

and

$a={\mu }_y-{\mu }_x·b={\mu }_y-{\mu }_x·\rho ·\frac{{\sigma }_y}{{\sigma }_x}$

Step 4: Final Answer

Minimize the expected squared error. Hence, we have proved the claimed

$a={\mu }_y-{\mu }_x·b={\mu }_y-{\mu }_x·\rho ·\frac{{\sigma }_y}{{\sigma }_x}$