Problem 18 - A Modern Introduction to Probability and Statistics: Understanding Why and How

Question

Suppose a vase contains balls numbered \(1,2, \ldots, N\). We draw \(n\) balls without replacement from the vase. Each ball is selected with equal probability, i.e., in the first draw each ball has probability \(1 / N\), in the second draw each of the \(N-1\) remaining balls has probability \(1 /(N-1)\), and so on. For \(i=\) \(1,2, \ldots, n\), let \(X_{i}\) denote the number on the ball in the \(i\) th draw. We have shown that the marginal probability mass function of \(X_{i}\) is given by $$ p_{X_{i}}(k)=\frac{1}{N}, \quad \text { for } \quad k=1,2, \ldots, N . $$ a. Show that $$ \mathrm{E}\left[X_{i}\right]=\frac{N+1}{2} $$ b. Compute the variance of \(X_{i}\). You may use the identity $$ 1+4+9+\cdots+N^{2}=\frac{1}{6} N(N+1)(2 N+1) $$

Step-by-Step Solution

Answer

\( \mathrm{E}[X_i] = \frac{N+1}{2} \) and \( \mathrm{Var}(X_i) = \frac{N^2-1}{12} \).

Step 1: Understand Expectation Formula

The expectation of a random variable is calculated using the formula \( \mathrm{E}[X] = \sum_{k=1}^{N} k \cdot p(k) \), where \( p(k) \) is the probability mass function of \( X_i \). For this problem, \( p_{X_i}(k) = \frac{1}{N} \).

Step 2: Apply Expectation Formula

Substitute \( p_{X_i}(k) = \frac{1}{N} \) into the expectation formula: \[ \mathrm{E}[X_i] = \sum_{k=1}^{N} k \cdot \frac{1}{N} = \frac{1}{N} \sum_{k=1}^{N} k. \]

Step 3: Calculate the Sum of First N Numbers

The sum of the first \( N \) natural numbers is given by \( \sum_{k=1}^{N} k = \frac{N(N+1)}{2} \). Substitute this into the expression for expectation: \[ \mathrm{E}[X_i] = \frac{1}{N} \cdot \frac{N(N+1)}{2} = \frac{N+1}{2}. \]

Step 4: Confirm Expectation Result

Thus, we have shown \( \mathrm{E}[X_i] = \frac{N+1}{2} \), which matches the given expression for the expectation.

Step 5: Understand Variance Formula

Variance is calculated using the formula \( \mathrm{Var}(X) = \mathrm{E}[X^2] - (\mathrm{E}[X])^2 \). We need to compute \( \mathrm{E}[X^2] \) first.

Step 6: Calculate Expected Value of Squares

Compute \( \mathrm{E}[X_i^2] = \sum_{k=1}^{N} k^2 \cdot \frac{1}{N} = \frac{1}{N} \cdot \sum_{k=1}^{N} k^2 \). Using the identity \( \sum_{k=1}^{N} k^2 = \frac{1}{6} N(N+1)(2N+1) \), we find \[ \mathrm{E}[X_i^2] = \frac{1}{N} \cdot \frac{1}{6} N(N+1)(2N+1) = \frac{1}{6} (N+1)(2N+1). \]

Step 7: Calculate Variance

Substitute into the variance formula: \[ \mathrm{Var}(X_i) = \mathrm{E}[X_i^2] - (\mathrm{E}[X_i])^2 = \frac{1}{6} (N+1)(2N+1) - \left(\frac{N+1}{2}\right)^2. \] Simplifying gives \[ \mathrm{Var}(X_i) = \frac{(N+1)(2N+1)}{6} - \frac{(N+1)^2}{4}. \]

Step 8: Simplify Variance Expression

Simplify the variance expression by finding a common denominator: \[ \mathrm{Var}(X_i) = \frac{2(N+1)(2N+1) - 3(N+1)^2}{12}. \] Factor and simplify further to get \[ \mathrm{Var}(X_i) = \frac{N^2-1}{12}. \]

Key Concepts

Expectation

Expectation, often referred to as the "expected value," is a crucial concept in probability and statistics. It provides the average or mean value that you would expect to obtain if the random experiment were repeated numerous times. For a random variable, the expectation is calculated by weighting each possible outcome by its probability and summing these values. In our exercise, we consider a random variable, say \(X_i\), which represents the number on the ball drawn from a vase. Every number from 1 to \(N\) has an equal chance of being drawn. Thus, the probability mass function (PMF) is uniform, with each event having a probability of \(\frac{1}{N}\).Mathematically, expectation is given by:- The formula \( \mathrm{E}[X] = \sum_{k=1}^{N} k \cdot p(k) \).- For our problem, substituting \( p_{X_i}(k) = \frac{1}{N} \), the expectation formula becomes: \[ \mathrm{E}[X_i] = \frac{1}{N} \sum_{k=1}^{N} k. \]- By calculating \( \sum_{k=1}^{N} k = \frac{N(N+1)}{2} \), we deduce that \( \mathrm{E}[X_i] = \frac{N+1}{2} \).This tells us that, on average, the number drawn is centered around the midway point between 1 and \(N\).

Variance

Variance measures the spread of a random variable's outcomes around its mean. It provides valuable insights into the variability of the data points. In simple terms, variance indicates how much the values deviate from the expectation (mean).For our exercise, the variance of a random variable \(X_i\) is calculated using the formula:- \( \mathrm{Var}(X_i) = \mathrm{E}[X_i^2] - (\mathrm{E}[X_i])^2 \). To find \( \mathrm{E}[X_i^2] \), the expected value of the square, we use:- \( \mathrm{E}[X_i^2] = \frac{1}{N} \sum_{k=1}^{N} k^2 \),- With the identity \( \sum_{k=1}^{N} k^2 = \frac{1}{6} N(N+1)(2N+1) \).- This leads to \( \mathrm{E}[X_i^2] = \frac{1}{6} (N+1)(2N+1). \)The variance formula now substitutes \( \mathrm{E}[X_i^2] \) as:- \( \mathrm{Var}(X_i) = \frac{1}{6} (N+1)(2N+1) - \left(\frac{N+1}{2}\right)^2. \)- With some simplification, the variance resolves to \( \mathrm{Var}(X_i) = \frac{N^2-1}{12}. \)This result informs us about how unpredictably widespread the numbers on the drawn balls can be, giving insights into the variation within our sample.

Random Variable

A random variable is a fundamental concept in probability, acting as a variable whose potential values result from the outcomes of a random phenomenon. In essence, it maps each outcome of an experiment to a numerical value.In our scenario:- Each draw from the vase represents a random experiment.- The number on the ball drawn is the outcome expressed as a random variable.These variables, say \(X_1, X_2, \ldots, X_n\), are used to evaluate subsequent statistical measures like expectation and variance. Each \(X_i\) is associated with an individual draw, and together they characterize multiple probabilities and statistical behaviors of our drawing process.

Probability Mass Function

A Probability Mass Function (PMF) is a function that gives the probability that a discrete random variable is exactly equal to some value. For discrete variables like in our problem, the PMF defines the likelihood of each specific value occurring.Our problem uses a PMF because:- We are dealing with discrete outcomes — numbers on drawn balls.- The function is uniform, meaning each number has the same chance of appearing.Mathematically for our exercise,- The PMF \(p_{X_i}(k)\) equals \(\frac{1}{N}\) for each integer \(k\) between 1 and \(N\).- This uniform distribution is characteristic of a simple random sample without replacement, where all outcomes are equally probable.Understanding the PMF is instrumental, as it lays the foundation for calculating more complex properties like expectation and variance, helping describe the overall behavior of the drawing process from our vase.