Q.7.38 - A First Course in Probability

Question

The best linear predictor of $Y$with respect to$X_1$ and $X_2$ is equal to $a + b{X}_1 + c{X}_2$, where $a$, $b$, and $c$ are chosen to minimize $E\left[{\left(Y-\left(a+b{X}_1+c{X}_2\right)\right)}^2\right]$ Determine $a$, $b$, and $c$. 

Step-by-Step Solution

Answer

$a=E\left(Y\right)-bE\left(X_1\right)-cE\left(X_2\right)$

$b=\frac{Cov\left(X_1,X_2\right)Cov\left(Y,X_2\right)-Cov\left(Y,X_1\right)Var\left(X_2\right)}{Cov{\left(X_1,X_2\right)}^2-Var\left(X_1\right)Var\left(X_2\right)}$

$c=\frac{Cov\left(X_1,X_2\right)Cov\left(Y,X_2\right)-Cov\left(Y,X_2\right)Var\left(X_1\right)}{Cov{\left(X_1,X_2\right)}^2-Var\left(X_1\right)Var\left(X_2\right)}$

Step 1: Given Information

The best linear predictor of $Y$ with respect to $X_1$ and $X_2$ is equal to $a + b{X}_1 + c{X}_2$.

Step 2: Explanation

Let's suppose that $Y,X_1,X_2$ are random variables $\Omega \to \mathrm{ℝ}$, where$(\Omega ,\mathcal{F},P)$ is the probability space. In that case, we have that

$E\left[{\left(Y-\left(a+b{X}_1+c{X}_2\right)\right)}^2\right]={\int }_{\Omega }{\left(Y-a-b{X}_1-c{X}_2\right)}^{2}dP$

Applying partial derivation of that expression respective to $a$

$\frac{\partial }{\partial a}E\left[{\left(Y-\left(a+b{X}_1+c{X}_2\right)\right)}^2\right]=\frac{\partial }{\partial a}{\int }_{\Omega }{\left(Y-a-b{X}_1-c{X}_2\right)}^{2}dP$

$={\int }_{\Omega }\frac{\partial }{\partial a}{\left(Y-a-b{X}_1-c{X}_2\right)}^{2}dP$

$=-2{\int }_{\Omega }\left(Y-a-b{X}_1-c{X}_2\right)dP$

$=-2E\left(Y-a-b{X}_1-c{X}_2\right)$

Step 3: Explanation

With respect to $b$,

$\frac{\partial }{\partial b}E\left[{\left(Y-\left(a+b{X}_1+c{X}_2\right)\right)}^2\right]=\frac{\partial }{\partial b}{\int }_{\Omega }{\left(Y-a-b{X}_1-c{X}_2\right)}^{2}dP$

$={\int }_{\Omega }\frac{\partial }{\partial b}{\left(Y-a-b{X}_1-c{X}_2\right)}^{2}dP$

$=-2{\int }_{\Omega }\left(Y-a-b{X}_1-c{X}_2\right)X_{1}dP$

$=-2E\left(\left(Y-a-b{X}_1-c{X}_2\right)X_1\right)$

Step 4: Explanation

$=-2{\int }_{\Omega }\left(Y-a-b{X}_1-c{X}_2\right)X_{2}dP$With respect to $c$,

$\frac{\partial }{\partial c}E\left[{\left(Y-\left(a+b{X}_1+c{X}_2\right)\right)}^2\right]=\frac{\partial }{\partial c}{\int }_{\Omega }{\left(Y-a-b{X}_1-c{X}_2\right)}^{2}dP$

$={\int }_{\Omega }\frac{\partial }{\partial c}{\left(Y-a-b{X}_1-c{X}_2\right)}^{2}dP$

$=-2E\left(\left(Y-a-b{X}_1-c{X}_2\right)X_2\right)$

Step 5: Explanation

Setting these partial derivations equal to zero gives us conditions

$E\left(Y\right)=a+bE\left(X_1\right)+cE\left(X_2\right)$

$E\left(Y{X}_1\right)=aE\left(X_1\right)+bE\left(X_1^2\right)+cE\left(X_1{X}_2\right)$

$E\left(Y{X}_2\right)=aE\left(X_2\right)+bE\left(X_1{X}_2\right)+cE\left(X_2^2\right)$

Step 6: Final Answer

Implies that,

$a=E\left(Y\right)-bE\left(X_1\right)-cE\left(X_2\right)$

$b=\frac{Cov\left(X_1,X_2\right)Cov\left(Y,X_2\right)-Cov\left(Y,X_1\right)Var\left(X_2\right)}{Cov{\left(X_1,X_2\right)}^2-Var\left(X_1\right)Var\left(X_2\right)}$

$c=\frac{Cov\left(X_1,X_2\right)Cov\left(Y,X_1\right)-Cov\left(Y,X_2\right)Var\left(X_1\right)}{Cov{\left(X_1,X_2\right)}^2-Var\left(X_1\right)Var\left(X_2\right)}$