Q.7.17 - A First Course in Probability

Question

Suppose that $X_1$ and $X_2$ are independent random variables having a common mean $\mu$. Suppose also that $Var\left(X_1\right)={\sigma }_1^2$ and $Var\left(X_2\right)={\sigma }_2^2$. The value of $\mu$ is unknown, and it is proposed that $\mu$ be estimated by a weighted average of $X_1$ and $X_2$. That is, $\lambda X_1+(1-\lambda )X_2$ will be used as an estimate of $\mu$ for some appropriate value of $\lambda$. Which value of $\lambda$ yields the estimate having the lowest possible variance? Explain why it is desirable to use this value of $\lambda .$

Step-by-Step Solution

Answer

Reason for it is desirable to use this value of $\lambda :$

As $\mathrm{Var}\left[\lambda X_1+(1-\lambda )X_2\right]=E\left[{\left(\lambda X_1+(1-\lambda )X_2\right)}^2\right]$ then $\lambda$ is to be small.

Step 1: Given Information

Independent Random variables $=X_1,X_2$

$Var\left(X_1\right)={\sigma }_1^2$

$Var\left(X_2\right)={\sigma }_2^2$

Value of $\mu =?$

Variance of $\lambda X_1+(1-\lambda )X_2=?$

Step 2: Explanation

Suppose that $X_1$ and $X_2$ are independent random variables having a common mean $\mu$. Let $\lambda X_1+(1-\lambda )X_2$ will be used as an estimate of $\mu$ for some appropriate value of $\lambda$. It is known that $Var\left(X_1\right)={\sigma }_1^2$ and $Var\left(X_2\right)={\sigma }_2^2$.

Find the variance of $\lambda X_1+(1-\lambda )X_2$.

$\mathrm{Var}\left(\lambda X_1+(1-\lambda )X_2\right)=\mathrm{Var}\left(\lambda X_1\right)+\mathrm{Var}\left[(1-\lambda )X_2\right]$

$={\lambda }^2\mathrm{Var}\left(X_1\right)+(1-\lambda {)}^2\mathrm{Var}\left[X_2\right]$

$={\lambda }^2{\sigma }_1^2+(1-\lambda {)}^2{\sigma }_2^2$

Step 3: Explanation

Find the value of $\lambda$ yields the estimate having the lowest possible variance? Differentiate with respect to $\lambda$ and then equating to $0.$

$\frac{d}{d\lambda }\left[{\lambda }^2{\sigma }_1^2+(1-\lambda {)}^2{\sigma }_2^2\right]=0$

$2\lambda {\sigma }_1^2-2(1-\lambda ){\sigma }_2^2=0$

$\lambda =\frac{{\sigma }_2^2}{{\sigma }_1^2+{\sigma }_2^2}$

Step 3: Final Answer

Therefore, 

Explain it is desirable to use this value of $\lambda .$

As$Var\left[\lambda X_1+(1-\lambda )X_2\right]=E\left[{\left(\lambda X_1+(1-\lambda )X_2\right)}^2\right]$, then $\lambda$is to be small.