Problem 16 - A Modern Introduction to Probability and Statistics: Understanding Why and How

Question

In this exercise we derive a kind of Jensen inequality for the minimum. a. Let \(a\) and \(b\) be real numbers. Show that $$ \min \\{a, b\\}=\frac{1}{2}(a+b-|a-b|) . $$ b. Let \(X\) and \(Y\) be independent random variables with the same distribution and finite expectation. Deduce from a that $$ \mathrm{E}[\min \\{X, Y\\}]=\mathrm{E}[X]-\frac{1}{2} \mathrm{E}[|X-Y|] $$ c. Show that $$ \mathrm{E}[\min \\{X, Y\\}] \leq \min \\{\mathrm{E}[X], \mathrm{E}[Y]\\} $$ Remark: this is not so interesting, since \(\min \\{\mathrm{E}[X], \mathrm{E}[Y]\\}=\mathrm{E}[X]=\mathrm{E}[Y]\), but we will see in the exercises of Chapter 11 that this inequality is also true for \(X\) and \(Y\), which do not have the same distribution.

Step-by-Step Solution

Answer

Jensen's inequality for minima is derived through expressing minimum of values using an absolute difference, and expectation properties. E[\min{X,Y}] = E[X] - 0.5E[|X-Y|], which adheres to E[\min{X,Y}] \leq \min{E[X],E[Y]}. E[X] = E[Y] by assumption here.

Step 1: Expression for Minimum of Two Numbers

To show that \( \min \{a, b\} = \frac{1}{2}(a+b-|a-b|) \), we start by considering the definition of minimum. If \(a \leq b\), then \(\min\{a, b\} = a\), and if \(b < a\), then \(\min\{a, b\} = b\). To express the minimum in a single formula, note that \(|a-b| = a-b\) when \(a \geq b\) and \(|a-b| = b-a\) when \(a < b\). Substitute \(|a-b|\) accordingly into the expression \(\frac{1}{2}(a+b-|a-b|)\):- If \(a \geq b\), we have: \(\frac{1}{2}(a+b-(a-b)) = \frac{1}{2}(2b) = b\).- If \(a < b\), we have: \(\frac{1}{2}(a+b-(b-a)) = \frac{1}{2}(2a) = a\).In both cases, \(\frac{1}{2}(a+b-|a-b|)\) gives the result of \(\min\{a, b\}\).

Step 2: Apply to Independent Random Variables

Given that \(X\) and \(Y\) are independent random variables with the same distribution and finite expectation, use the result from part a: \(\min\{X, Y\} = \frac{1}{2}(X+Y-|X-Y|)\). Take the expectation of both sides:\[ \mathrm{E}[\min \{X, Y\}] = \mathrm{E}\left[\frac{1}{2}(X+Y-|X-Y|)\right] \].Using linearity of expectation:\[ \mathrm{E}[\min \{X, Y\}] = \frac{1}{2}(\mathrm{E}[X] + \mathrm{E}[Y] - \mathrm{E}[|X-Y|]) \].Since \(X\) and \(Y\) have the same distribution, \(\mathrm{E}[X] = \mathrm{E}[Y]\), thus:\[ \mathrm{E}[\min \{X, Y\}] = \mathrm{E}[X] - \frac{1}{2} \mathrm{E}[|X-Y|] \].

Step 3: Derive Jensen Inequality for Minima

Using the result from Step 2, \( \mathrm{E}[\min \{X, Y\}] = \mathrm{E}[X] - \frac{1}{2} \mathrm{E}[|X-Y|] \), we need to show that this is \(\leq \min \{\mathrm{E}[X], \mathrm{E}[Y]\}\). Since \(\mathrm{E}[X] = \mathrm{E}[Y]\) by assumption, this inequality simplifies to:\[ \mathrm{E}[\min \{X, Y\}] \leq \mathrm{E}[X] \].The term \(\frac{1}{2} \mathrm{E}[|X-Y|]\) is non-negative, as expectation of absolute values is always non-negative, fulfilling the Jensen Inequality for Minima under given conditions.

Key Concepts

Expectation of Random Variables

The expectation of a random variable is a fundamental concept in probability and statistics. It is often associated with the 'average' or 'mean' value that a random variable can assume. When we are dealing with a random variable, which essentially is a mathematical object representing uncertainty, the expectation gives us a single number that characterizes its central tendency.
Let's consider a random variable, denoted as \(X\). The expectation of this random variable, denoted as \(\mathrm{E}[X]\), is calculated based on the probability distribution of \(X\). If \(X\) is a discrete random variable, the expectation is computed as:

• \(\mathrm{E}[X] = \sum x_i P(X = x_i)\), where \(x_i\) are the possible values of \(X\) and \(P(X = x_i)\) is the probability of \(X\) taking the value \(x_i\).

For continuous random variables, the expectation involves an integral:

• \(\mathrm{E}[X] = \int x f(x) dx\), where \(f(x)\) is the probability density function of \(X\).

Understanding the expectation is crucial when dealing with random variables, as it serves as a key building block for more complex concepts like variance, covariance, and other statistical measures. Thus, in scenarios involving independent random variables like \(X\) and \(Y\), knowing their expectations can help us deduce other properties, such as those highlighted with Jensen's Inequality.

Independent Random Variables

Independent random variables play a crucial role in statistical analysis. Two random variables, \(X\) and \(Y\), are said to be independent if the occurrence of one does not affect the probability of the occurrence of the other. This means that whatever happens with one variable does not sway our expectations or probabilities regarding the other.
Mathematically, this independence is expressed as:

• \(P(X = x, Y = y) = P(X = x) \cdot P(Y = y)\) for all possible values \(x\) and \(y\).

The property of independence simplifies many calculations, particularly when dealing with expectations. For example, the expectation of the sum of two independent random variables \(X\) and \(Y\) is simply the sum of their expectations:

• \(\mathrm{E}[X + Y] = \mathrm{E}[X] + \mathrm{E}[Y]\)

In the exercise in question, \(X\) and \(Y\) are independent and identically distributed, meaning they have the same expectation. This symmetry and independence are cleverly exploited when applying Jensen's Inequality to draw conclusions about the expected minimum of \(X\) and \(Y\). This exemplifies how independence can significantly streamline complex probability calculations.

Absolute Value Function

The absolute value function is a simple yet powerful mathematical tool. When dealing with inequalities and expectations, especially in the context of random variables, the absolute value can quantify the magnitude of deviations or distances without regard to direction.
The absolute value of a number \(a\) is denoted as \(|a|\) and is defined as:

• \(|a| = a \) if \(a \geq 0\)
• \(|a| = -a \) if \(a < 0\)

In the context of the given exercise, the expression \(|X - Y|\) represents the absolute difference between two random variables \(X\) and \(Y\). Considering their expected absolute difference, i.e., \(\mathrm{E}[|X-Y|]\), is insightful because it provides a sense of how much, on average, these variables deviate from each other.
This absolute difference is crucial in deriving certain inequalities, such as the modified Jensen's Inequality for minimum, where the expectation of this absolute value acts almost like a 'penalty' or adjustment to the general expectation, revealing fine insights into stochastic relationships between variables. Understanding and employing the absolute value function, therefore, is key to manipulating and interpreting these probabilistic expressions accurately.