Properties of Expectation

Suppose that $X_1$ and $X_2$ are independent random variables having a common mean $\mu$. Suppose also that $Var\left(X_1\right)={\sigma }_1^2$ and $Var\left(X_2\right)={\sigma }_2^2$. The value of $\mu$ is unknown, and it is proposed that $\mu$ be estimated by a weighted average of $X_1$ and $X_2$. That is, $\lambda X_1+(1-\lambda )X_2$ will be used as an estimate of $\mu$ for some appropriate value of $\lambda$. Which value of $\lambda$ yields the estimate having the lowest possible variance? Explain why it is desirable to use this value of $\lambda$

Show that $Y=a+b X$, then

$\rho (X,Y)=\left\{\begin{array}{l}+1\hspace{0.25em}\hspace{0.25em}\hspace{0.25em}\hspace{0.25em}\text{ if }b>0\\ -1\hspace{0.25em}\hspace{0.25em}\hspace{0.25em}\hspace{0.25em}\text{ if }b<0\end{array}\right.$

Show that $Z$ is a standard normal random variable and if $Y$ is defined by $Y=a+bZ+c{Z}^2$, then

$\rho (Y,Z)=\frac{b}{\sqrt{b^2+2c^2}}$

Prove the Cauchy-Schwarz inequality, namely,

$\left(E\right[XY]{)}^2\le E\left[X^2\right]E\left[Y^2\right]$

Hint: Unless$Y=-t X$ for some constant, in which case the inequality holds with equality, it follows that for all $t$,

$0<E\left[(tX+Y{)}^2\right]=E\left[X^2\right]t^2+2E\left[XY\right]t+E\left[Y^2\right]$

Hence, the roots of the quadratic equation

$E\left[X^2\right]t^2+2E\left[XY\right]t+E\left[Y^2\right]=0$

must be imaginary, which implies that the discriminant of this quadratic equation must be negative.

7.34. For another approach to Theoretical Exercise 7.33, let Tr denote the number of flips required to obtain a run of r consecutive heads. (a) Determine E[Tr|Tr−1]. (b) Determine  in terms of E[Tr−1]. (c) What is E[T1]? (d) What is E[Tr]? 

The probability generating function of the discrete non-negative integer-valued random variable $X$ having probability mass function $p_j,j\ge 0$is defined by

$\varphi \left(s\right)=E\left[s^X\right]=\sum _{j=0}^{\infty }{p}_j{s}^j$

Let $Y$ be a geometric random variable with parameter $p=1-s$, where $0<s<1$. Suppose that $Y$ is independent of $X$, and show that
$\varphi \left(s\right)=P\{X<Y\}$

 For an event A, let IA equal 1 if A occurs and let it equal 0 if A does not occur. For a random variable X, show that E[X|A] = E[XIA] P(A 

A coin that lands on heads with probability p is continually flipped. Compute the expected number of flips that are made until a string of r heads in a row is obtained. Hint: Condition on the time of the first occurrence of tails to obtain the equation E[X] = (1 − p) r i=1 pi−1(i + E[X]) +(1 − p) q i=r+1 pi−1r Simplify and solve for E[X]. 

The best linear predictor of $Y$with respect to$X_1$ and $X_2$ is equal to $a + b{X}_1 + c{X}_2$, where $a$, $b$, and $c$ are chosen to minimize $E\left[{\left(Y-\left(a+b{X}_1+c{X}_2\right)\right)}^2\right]$ Determine $a$, $b$, and $c$. 

Consider Example 4f, which is concerned with the multinomial distribution. Use conditional expectation to compute E[NiNj], and then use this to verify the formula for Cov(Ni, Nj) given in Example 4f. 

For another approach to Theoretical Exercise 7.33, let Tr denote the number of flips required to obtain a run of r consecutive heads. 

(a) Determine $E\left[T_r\mid T_{r-1}\right]$.

(b) Determine $E\left[T_r\right]$ in terms of $E\left[T_{r-1}\right]$.

(c) What is $E\left[T_1\right]$ ?

(d) What is $E\left[T_r\right]$ ?

One ball at a time is randomly selected from an urn containing a white and b black balls until all of the remaining balls are of the same color. Let Ma,b denote the expected number of balls left in the urn when the experiment ends. Compute a recursive formula for Ma,b and solve when a = 3 and b = 5. 

An urn contains$a$ white and $b$black balls. After a ball is drawn, it is returned to the urn if it is white; but if it is black, it is replaced by a white hall from another urn. Let $M_n$ denote the expected number of white balls in the urn after the foregoing operation has been repeated$n$ times.

1.  Derive the recursive equation $M_{n+1}=\left(1-\frac{1}{a+b}\right)M_n+1$
   
2. Use part (a) to prove that $M_n=a+b-b{\left(1-\frac{1}{a+b}\right)}^n$
   
3. What is the probability that the $(n+1)$ ball drawn is white?

Show that if $X$ and $Y$ are independent, then

$E[X\mid Y=y]=E\left[X\right] \text{ for all }y$

(a) in the discrete case;

(b) in the continuous case.

Prove that $E\left[g\right(X)Y\mid X]=g\left(X\right)E[Y\mid X]$.

Prove that if $E[Y\mid X=x]=E\left[Y\right]$ for all $x$, then $X$ and $Y$ are uncorrelated; give a counterexample to show that the converse is not true.

Hint: Prove and use the fact that $E\left[XY\right]=E\left[XE\right[Y\mid X\left]\right]$.

Show that $Cov(X,E[Y\mid X\left]\right)=Cov(X,Y)$.

Let $X_1,\dots ,X_n$ be independent and identically distributed random variables. Find

$E\left[X_1\mid X_1+\cdots +X_n=x\right]$

An urn initially contains $b$ black and $w$ white balls. At each stage, we add $r$ black balls and then withdraw, at random,$r$ balls from the $b+w+r$ balls in the urn. Show that

$F$ [number of white halls after stage $t$ ]

$={\left(\frac{b+w}{b+w+r}\right)}^{t}w$

The best quadratic predictor of$Y$with respect to $X$ is a + b$X + c{X}_2$, where a, b, and c are chosen to minimize $E\left[\right(Y - (a + bX + c{X}_2)\left)2\right]$. Determine $a$, $b$, and $c$. 

Use the conditional variance formula to determine the variance of a geometric random variable $X$ having parameter $p$.

It follows from Proposition $6.1$ and the fact that the best linear predictor of $Y$ with respect to $X$ is $\mu y + \rho \sigma y/\sigma x (X - \mu x)$ that if $E\left[Y\right|X] = a + bX$ then $a = \mu y - \rho \sigma y/\sigma x \mu x$ $b = \rho \sigma y \sigma x$ (Why?) Verify this directly 

Show that for random variables $X$ and $Z$.

$E\left[(X-Y{)}^2\right]=E\left[X^2\right]-E\left[Y^2\right]$

where $Y=E[X\mid Z]$

For a standard normal random variable $Z$,

Show that ${\mu }_n=\left\{\begin{array}{l}0\\ \frac{\left(2j\right)! }{2jj! }\end{array}\right.$

Hint: Start by expanding the moment generating function of $Z$ into a Taylor series about $0$ to obtain

$E\left[e^{tZ}\right]=e^{t^2/2}$

$=\sum _{j=0}^{\infty }\frac{{\left(t^2/2\right)}^j}{j!}$

Let X be a normal random variable with mean $\mu$ and variance ${\sigma }^2$. Use the results of Theoretical Exercise 7.46 to show that

$E\left[X^n\right]=\sum _{j=0}^{[n/2]} \frac{\left(\begin{array}{c}n\\ 2j\end{array}\right){\mu }^{n-2j}{\sigma }^{2j}\left(2j\right)!}{2jj!}$

In the preceding equation, $[n/2]$ is the largest integer less than or equal to $n/2$. Check your answer by letting $n = 1$ and $n = 2$. 

Verify the formula for the moment generating function of a uniform random variable that is given in Table 7.2. Also, differentiate to verify the formulas for the mean and variance. 

Consider a population consisting of individuals able to produce offspring of the same kind. Suppose that by the end of its lifetime, each individual will have produced j new offspring with probability P_j, $j\ge 0$, independently of the number produced by any other individual. The number of individuals initially present, denoted by X₀, is called the size of the zeroth generation. All offspring of the zeroth generation constitute the first generation, and their number is denoted by X1. In general, let X_n denote the size of the nth generation. Let $\mu =\sum _{j=0}^{\mathrm{\infty }} j{P}_j$ and ${\sigma }^2=\sum _{j=0}^{\mathrm{\infty }} (j-\mu {)}^2{P}_j$ denote, respectively, the mean and the variance of the number of offspring produced by a single individual. Suppose that X₀ = 1— that is, initially there is a single individual in the population 

(a) Show that $E\left[X_n\right]=\mu E\left[X_{n-1}\right]$.

(b) Use part (a) to conclude that $E\left[X_n\right]={\mu }^n$

(c) Show that $\mathrm{Var}\left(X_n\right)={\sigma }^2{\mu }^{n-1}+{\mu }^2\mathrm{Var}\left(X_{n-1}\right)$

(d) Use part (c) to conclude that 

$\mathrm{Var}\left(X_n\right)=\left\{\begin{array}{l}{\sigma }^2{\mu }^{n-1}\left(\frac{{\mu }^n-1}{\mu -1}\right)\hspace{0.25em}\hspace{0.25em}\hspace{0.25em}\hspace{0.25em}\text{ if }\mu \ne 1\\ n{\sigma }^2\hspace{0.25em}\hspace{0.25em}\hspace{0.25em}\hspace{0.25em}\text{ if }\mu =1\end{array}\right.$

The model just described is known as a branching process, and an important question for a population that evolves along such lines is the probability that the population will eventually die out. Let π denote this probability when the population starts with a single individual. That is, 

$\left.\pi =P\text{ {population eventually dies out }\mid X_0=1\right)$

(e) Argue that π satisfies 

$\pi =\sum _{j=0}^{\mathrm{\infty }} P_j{\pi }^j$

Let X be a normal random variable with parameters μ = 0 and σ2 = 1, and let I, independent of X, be such that P{I = 1} = 1 2 = P{I = 0}.  Now define Y by Y = X if I = 1 −X if I = 0 In words, Y is equally likely to equal either X or $-X.$

(a) Are X and Y independent? 

(b) Are I and Y independent? 

(c) Show that Y is normal with mean $0$ and variance $1$.

(d) Show that $Cov (X, Y) = 0$

An urn has n white and m black balls that are removed one at a time in a randomly chosen order. Find the expected number of instances in which a white ball is immediately followed by a black one.

Twenty individuals consisting of $10$ married couples are to be seated at $5$ different tables, with $4$ people at each table.

 (a) If the seating is done“at random,” what is the expected number of married couples that are seated at the same table? 

(b) If $2$ men and $2$ women are randomly chosen to be seated at each table, what is the expected number of married couples that are seated at the same table?

If a die is to be rolled until all sides have appeared at least once, find the expected number of times that outcome $1$ appears.

A deck of $2$n cards consists of n red and n black cards. The cards are shuffled and then turned over one at a time. Suppose that each time a red card is turned over, we win 1 unit if more red cards than black cards have been turned over by that time. (For instance, if n = $2$ and the result is r b r b, then we would win a total of $2$ units.) Find the expected amount that we win.

Let $A_1,A_{2,\dots ,}{A}_n$ be events, and let $N$ denote the number of them that occur. Also, let $I=1$ if all of these events occur, and let it be $0$ otherwise. Prove Bonferroni’s inequality, namely,

$P(A_1\cdots A_n)\ge \sum _{i=1}^{n}P\left(A_i\right)-(n-1)$

Hint: Argue first that $N\le n-1+I$

Suppose that X₁, ... , X_n have a multivariate normal distribution. Show that X₁, ... , X_n are independent random variables if and only if 

$Cov\left(X_i,X_j\right)=0$ when $i\ne j$

If Z is a standard normal random variable, what is Cov(Z, Z²)? 

Suppose that Y is a normal random variable with mean μ and variance ${\sigma }^2$, and suppose also that the conditional distribution of X, given that Y = y, is normal with mean y and variance 1.

(a) Argue that the joint distribution of X, Y is the same as that of $Y + Z$, Y when Z is a standard normal random variable that is independent of Y.

(b) Use the result of part (a) to argue that X, Y has a bivariate normal distribution. 

(c) Find E[X], Var(X), and Corr(X, Y). (d) Find E[Y|X = x]. 

(e) What is the conditional distribution of Y given that X = x? 

7.1. Consider a list of m names, where the same name may appear more than once on the list. Let $n\left(i\right), i = 1, ... , m$, denote the number of times that the name in position i appears on the list, and let d denote the number of distinct names on the list. 

(a) Express d in terms of the variables $m, n\left(i\right), i = 1, ... , m$. Let U be a uniform (0, 1) random variable, and let $X = \left[mU\right] + 1$. 

(b) What is the probability mass function of X? 

(c) Argue that $E[m/n(X\left)\right] = d$. 

If $Y=a X+b$ where a and b are constants, express the moment generating function of $Y$ in terms of the moment generating function of $X$. 

The positive random variable $X$ is said to be a lognormal random variable with parameters $\mu$ and${\sigma }^2$ if $\log \left(X\right)$ is a normal random variable with mean $\mu$and variance ${\sigma }^2$. Use the normal moment generating function to find the mean and variance of a lognormal random variable 

Let $X$ have moment generating function $M\left(t\right)$, and define$\Psi \left(t\right)=\log M\left(t\right)$. Show that ${\left.{\Psi }^{\text{'}\text{'}}\left(t\right)\right|}_{t=0}=Var\left(X\right)$.

Use Table $7.2$ to determine the distribution of $\sum _{i=1}^n{X}_i$ when $X_1,\dots ,X_n$are independent and identically distributed exponential random variables, each having mean$1/\lambda$.

Show how to compute $Cov(X,Y)$ from the joint moment generating function of$X$ and $Y$. 

Let $X$ be the smallest value obtained when $k$ numbers are randomly chosen from the set $1,\dots ,n$. Find $E\left[X\right]$ by interpreting $X$ as a negative hypergeometric random variable.

Suppose in Self-Test Problem $7.3$ that the $20$ people are to be seated at seven tables, three of which have $4$ seats and four of which have $2$ seats. If the people are randomly seated, find the expected value of the number of married couples that are seated at the same table.

Let $X$ be a nonnegative random variable having a distribution function $F$. Show that if $\overline{F}\left(x\right)=1-F\left(x\right)$, then

$E\left[X^n\right]={\int }_0^{\infty }{x}^{n-1}\overline{F}\left(x\right)dx$

Hint: Start with the identity

$X^n=n{\int }_0^X{x}^{n-1}dx$

$=n{\int }_0^{\infty }{x}^{n-1}{I}_X\left(x\right)dx$

where
$I_x\left(x\right)=\left\{\begin{array}{l}1\\ 0\end{array}\right.$if$x<X$ otherwise

Let X be the length of the initial run in a random ordering of n ones and m zeros. That is, if the first k values are the same (either all ones or all zeros), then X Ú k. Find E[X]. 

There are n items in a box labeled H and m in a box labeled T. A coin that comes up heads with probability p and tails with probability 1 − p is flipped. Each time it comes up heads, an item is removed from the H box, and each time it comes up tails, an item is removed from the T box. (If a box is empty and its outcome occurs, then no items are removed.) Find the expected number of coin flips needed for both boxes to become empty. Hint: Condition on the number of heads in the first n + m flips.

A total of$m$ items are to be sequentially distributed among $n$ cells, with each item independently being put in a cell $j$ with probability $pj, j = 1, ...,n$ . Find the expected number of collisions that occur, where a collision occurs whenever an item is put into a non-empty cell.

Suppose in Self-Test Problem $7.3$ that the $20$ people are to be seated at seven tables, three of which have $4$ seats and four of which have $2$ seats. If the people are randomly seated, find the expected value of the number of married couples that are seated at the same table. 

Individuals $1$ through n,n > $1$, are to be recruited into a firm in the following manner: Individual $1$ starts the firm and recruits individual $2$. Individuals $1$ and $2$ will then compete to recruit individual $3$. Once individual $3$ is recruited, individuals $1,2$, and $3$ will compete to recruit individual $4$, and so on. Suppose that when individuals $1,2$,...,i compete to recruit individual i + $1$, each of them is equally likely to be the successful recruiter.

(a) Find the expected number of the individuals $1$,...,n who did not recruit anyone else.

(b) Derive an expression for the variance of the number of individuals who did not recruit anyone else, and evaluate it for n=$5$.