Problem 10 - A First Course in the Numerical Analysis of Differential Equations

Question

Let \(B\) be a \(d \times d\) normal matrix and let \(y \in \mathbb{C}^{a}\) be an arbitrary vector such that \(\|\boldsymbol{y}\|=1\) (in the Euclidean norm). a Prove that there exist numbers \(\alpha_{1}, \alpha_{2}, \ldots, \alpha_{d}\) such that \(y=\sum_{k=1}^{d} \alpha_{k} w_{k}\), where \(w_{1}, w_{2}, \ldots, w_{d}\) are the eigenvectors of \(B .\) Express \(\|y\|^{2}\) explicitly in terms of \(\alpha_{k}, k=1,2, \ldots, d\). \(\mathbf{b}\) Let \(\lambda_{1}, \lambda_{2}, \ldots, \lambda_{d}\) be the eigenvalues of \(B, B w_{k}=\lambda_{k} w_{k}, k=1,2, \ldots, d\). Prove that $$ \|B y\|^{2}=\sum_{k=1}^{d}\left|\alpha_{k} \lambda_{k}\right|^{2} $$ c Deduce that \(\|B\|=\rho(B)\).

Step-by-Step Solution

Answer

The vector \( y \) can be expressed in terms of eigenvectors of \( B \), and \( \|B\|=\rho(B) \).

Step 1: Orthogonal Decomposition

Since \( B \) is a normal \( d \times d \) matrix, it can be diagonalized by a unitary matrix. Hence, \( B \) has eigenvectors \( w_1, w_2, \ldots, w_d \) that form an orthonormal basis of \( \mathbb{C}^d \). An arbitrary vector \( y \in \mathbb{C}^{a} \) can be expressed as a linear combination of these eigenvectors: \( y = \sum_{k=1}^{d} \alpha_{k} w_{k} \). The coefficients \( \alpha_k \) are found using the inner product \( \alpha_k = \langle y, w_k \rangle \).

Step 2: Calculate Norm Using Coefficients

The norm of a vector \( y \) expressed in terms of its coefficients is \( \|y\|^2 = \langle y, y \rangle \). Substituting the expression from Step 1, \( \|y\|^2 = \left\langle \sum_{k=1}^{d} \alpha_{k} w_{k}, \sum_{j=1}^{d} \alpha_{j} w_{j} \right\rangle = \sum_{k=1}^{d} |\alpha_k|^2 \), because the eigenvectors form an orthonormal basis.

Step 3: Express \( \|By\|^2 \) in Terms of Eigenvectors and Eigenvalues

Since \( B w_k = \lambda_k w_k \), when \( y = \sum_{k=1}^{d} \alpha_k w_k \), we have \( B y = \sum_{k=1}^{d} \alpha_k B w_k = \sum_{k=1}^{d} \alpha_k \lambda_k w_k \). The norm is \( \|B y\|^2 = \langle B y, B y \rangle = \sum_{k=1}^{d} |\alpha_k \lambda_k|^2 \).

Step 4: Relate Matrix Norm to Spectral Radius

The spectral norm of \( B \), \( \|B\| \), is defined as \( \sup_{\|y\|=1} \|By\| \). From Step 3, \( \|By\|^2 = \sum_{k=1}^{d} |\alpha_k \lambda_k|^2 \), which is maximized when the largest eigenvalue module (spectral radius, \( \rho(B) \)) dominates. Therefore, \( \|B\| = \rho(B) \), confirming \( \|B y\| \leq \rho(B) \) with equality if \( y \) is an eigenvector associated with the eigenvalue of maximum magnitude.

Key Concepts

Spectral Radius

The spectral radius of a matrix is a fundamental concept in numerical analysis. It is defined as the largest absolute value of the eigenvalues of a matrix. When we talk about the spectral radius of a matrix \( B \), we denote it as \( \rho(B) \). This measure gives insight into the behavior of matrix transformations, particularly how they stretch or compress vectors in space.

Understanding the spectral radius is crucial because it provides the threshold for the stability of numerical methods. For instance, in iterative methods for solving linear systems, the spectral radius must be less than one for convergence. The spectral radius \( \rho(B) \) is found by computing all eigenvalues \( \lambda_1, \lambda_2, \ldots, \lambda_d \) of the matrix \( B \), and then taking the maximum of their magnitudes. In formal terms:

• \( \rho(B) = \max \{ |\lambda_1|, |\lambda_2|, \ldots, |\lambda_d| \} \)

This radius provides insight not just into stability, but also into the spectral properties which are important for diagonalization and orthogonal decomposition of matrices.

Eigenvalues and Eigenvectors

Eigenvalues and eigenvectors are key components in the study of linear algebra, forming the backbone of many numerical analysis techniques. When a matrix \( B \) acts on an eigenvector \( w_k \), the result is a scaled version of that vector, which is represented as \( B w_k = \lambda_k w_k \), where \( \lambda_k \) is the eigenvalue associated with the eigenvector \( w_k \).

This property allows eigenvalues and eigenvectors to simplify matrix operations such as finding powers of a matrix or solving differential equations. For normal matrices, the eigenvectors can form an orthonormal basis. This means any vector \( y \) in the vector space can be expressed as a linear combination of these eigenvectors. The coefficients of this linear combination are discovered by taking inner products with the eigenvectors:

• \( y = \sum_{k=1}^{d} \alpha_k w_k \)
• \( \alpha_k = \langle y, w_k \rangle \)

Understanding how to break down a matrix into its eigenvalues and eigenvectors helps in various numerical computations, like solving systems of equations or analyzing linear transformations.

Matrix Norms

Matrix norms give us a way to measure the size or 'magnitude' of a matrix. They're crucial for understanding the behavior of matrices in various mathematical contexts. One common type is the spectral norm, which relates directly to the eigenvalues of the matrix.

The spectral norm of matrix \( B \), denoted as \( \|B\| \), is computed by finding the maximum stretching factor of the matrix. Mathematically, this norm is defined in terms of the largest singular value or equivalently, the square root of the maximum eigenvalue of \( B^*B \), where \( B^* \) is the conjugate transpose of \( B \). However, in the exercise provided we learn that the spectral norm is equal to the spectral radius in particular cases:

• The norm \( \|B\| \) is calculated as \( \sup_{\|y\|=1} \|By\| \)
• This norm simplifies to the spectral radius \( \rho(B) \) if \( y \) is an eigenvector associated with an eigenvalue of maximum magnitude

This key fact simplifies numerous problems by reducing computations to analyzing the spectral radius and aids in predictions about the matrix's behavior, such as knowing the growth rate of transformed vectors.