Q. 7.17 - A First Course in Probability

Question

Suppose that $X_1$ and $X_2$ are independent random variables having a common mean $\mu$. Suppose also that $Var\left(X_1\right)={\sigma }_1^2$ and $Var\left(X_2\right)={\sigma }_2^2$. The value of $\mu$ is unknown, and it is proposed that $\mu$ be estimated by a weighted average of $X_1$ and $X_2$. That is, $\lambda X_1+(1-\lambda )X_2$ will be used as an estimate of $\mu$ for some appropriate value of $\lambda$. Which value of $\lambda$ yields the estimate having the lowest possible variance? Explain why it is desirable to use this value of $\lambda$

Step-by-Step Solution

Answer

Reason for it is desirable to use this value of $\lambda .$

As $Var\left[\lambda X_1+(1-\lambda )X_2\right]=E\left[{\left(\lambda X_1+(1-\lambda )X_2\right)}^2\right]$, then $\lambda$ is to be small.

Step 1: Given Information

Independent Random variables$=X_1,X_2$

$Var\left(X_1\right)={\sigma }_1^2$

$Var\left(X_2\right)={\sigma }_2^2$

$\mu =?$

$\lambda X_1+(1-\lambda )X_2$ will be used as an estimate of $\mu .$

Step 2: Explanation

Suppose that $X_1$ and $X_2$ are independent random variables having a common mean $\mu$.

Let $\lambda X_1+(1-\lambda )X_2$ will be used as an estimate of $\mu$ for some appropriate value of $\lambda$. It is known that $Var\left(X_1\right)={\sigma }_1^2$and $Var\left(X_2\right)={\sigma }_2^2$

Find the variance of $\lambda X_1+(1-\lambda )X_2,$

$\begin{array}{r}\mathrm{Var}\left(\lambda X_1+(1-\lambda )X_2\right)=\mathrm{Var}\left(\lambda X_1\right)+\mathrm{Var}\left[(1-\lambda )X_2\right]\end{array}$

$={\lambda }^2\mathrm{Var}\left(X_1\right)+(1-\lambda {)}^2\mathrm{Var}\left[X_2\right]$

$={\lambda }^2{\sigma }_1^2+(1-\lambda {)}^2{\sigma }_2^2$

Step 3: Explanation

Find the value of $\lambda$ yields the estimate having the lowest possible variance:

Differentiate with respect to $\lambda$ and then equating to $0.$

$\begin{array}{r}\frac{d}{d\lambda }\left[\mathrm{Var}\left(\lambda X_1+(1-\lambda )X_2\right)\right]=0\end{array}$

$\begin{array}{r}\frac{d}{d\lambda }\left[{\lambda }^2{\sigma }_1^2+(1-\lambda {)}^2{\sigma }_2^2\right]=0\end{array}$

$2\lambda {\sigma }_1^2-2(1-\lambda ){\sigma }_2^2=0$

$\lambda =\frac{{\sigma }_2^2}{{\sigma }_1^2+{\sigma }_2^2}$

Step 4: Final Answer

The reason for it is desirable to use this value of $\lambda ,$

As $Var\left[\lambda X_1+(1-\lambda )X_2\right]=E\left[{\left(\lambda X_1+(1-\lambda )X_2\right)}^2\right]$ then $\lambda$ is to be small.