Problem 31 - Calculus for Biology and Medicine

Question

We have two formulas for computing the variance of \(X\), namely, $$\operatorname{var}(X)=E[X-E(X)]^{2}$$ and $$\operatorname{var}(X)=E\left(X^{2}\right)-[E(X)]^{2}$$ (a) Explain why \(\operatorname{var}(X) \geq 0\). (b) Use your results in (a) to explain why $$ E\left(X^{2}\right) \geq[E(X)]^{2} $$

Step-by-Step Solution

Answer

Variance is non-negative, implying \( E[X^2] \geq [E(X)]^2 \).

Step 1: Understanding Variance Definition

The variance of a random variable \( X \), denoted as \( \operatorname{var}(X) \), is a measure of how spread out the values of the random variable are around its mean (expected value). The mathematical expectation or mean of a random variable \( X \) is denoted as \( E(X) \). Variance is defined as \( \operatorname{var}(X) = E[(X - E(X))^2] \).

Step 2: Non-negativity of Squared Terms

Consider the definition of variance \( \operatorname{var}(X) = E[(X - E(X))^2] \). The expression \( (X - E(X))^2 \) is always non-negative, since squaring any real number yields a non-negative result. Therefore, the expectation \( E[(X - E(X))^2] \) is also non-negative: \( \operatorname{var}(X) \geq 0 \).

Step 3: Simplifying Variance with Expectation Algebra

Variance can also be expressed as \( \operatorname{var}(X) = E[X^2] - [E(X)]^2 \). This form comes from expanding \( (X - E(X))^2 \) to \( X^2 - 2X \cdot E(X) + [E(X)]^2 \), and then taking the expectation, which simplifies to \( E[X^2] - [E(X)]^2 \).

Step 4: Implication of Non-negativity on Expectation

Since variance is non-negative (\( \operatorname{var}(X) \geq 0 \)), the alternative expression of variance \( \operatorname{var}(X) = E[X^2] - [E(X)]^2 \) implies that \( E[X^2] - [E(X)]^2 \geq 0 \). Rearranging gives us \( E[X^2] \geq [E(X)]^2 \). This result shows that the expectation of the square of the random variable \( X \) is no less than the square of its expectation.

Key Concepts

Expected Value

Understanding the concept of expected value makes it easier to comprehend various statistics measures, such as variance. The expected value, often denoted as \( E(X) \), is essentially the average of all possible values of a random variable, weighted by their probabilities. It acts as a central tendency measure, guiding us on what value we might anticipate from a future observation.

To find the expected value, for a discrete random variable, we sum up the products of each possible outcome and their probability, like so:

• \( E(X) = \sum{ x_i \, P(x_i) } \)

For continuous random variables, the expected value involves an integral:

• \( E(X) = \int{x \, f(x) \, dx} \)

This calculation provides a single value that summarizes the entire probability distribution, enabling further computations like variance.

Non-negative Variance

Variance is a powerful tool for measuring how dispersed the values of a random variable are. The simple yet crucial observation is that variance is always non-negative. But why is this so? This arises from the fact that variance is defined as \( E[(X - E(X))^2] \).

To understand this, note that \((X - E(X))^2\) represents squared deviations from the mean. Squaring a number, whether positive or negative, results in a non-negative quantity. Thus, \((X - E(X))^2\) is always at least zero.

When we take the expected value of these squared deviations, represented as \( E[(X - E(X))^2] \), we are essentially averaging them. The result is a non-negative number: variance cannot be negative. This property is fundamental to understanding many statistical concepts, ensuring computational consistency and logical results.

Expectation Algebra

Expectation algebra helps us manipulate and simplify expressions involving expectations, much like regular algebra does with numbers. Let's dig into how it simplifies the alternative formula for variance.

Given the expression \( (X - E(X))^2 \), we expand it using algebraic rules:

• \( (X - E(X))^2 = X^2 - 2X \, E(X) + [E(X)]^2 \)

Applying expectation algebra, the expected value of each part becomes:

• \( E[X^2] - 2E[X]E[X] + E[X]^2 \)

Notice \( E[X] = E(X) \), thus, the expression simplifies to \( E[X^2] - [E(X)]^2 \).

This demonstrates how variance, defined as \( E[(X - E(X))^2] \), is equivalent to another useful form \( E[X^2] - [E(X)]^2 \). By understanding and using these simplifications, expectation algebra becomes a critical tool in statistics.