Problem 21 - Advanced Linear Algebra

Question

Show that the tensor product is not, in general, commutative.

Step-by-Step Solution

Answer

The tensor product is not commutative in general, as illustrated by the following counterexample. Consider two two-dimensional column vectors: \( A = \begin{bmatrix} a_1 \\ a_2 \end{bmatrix} \) and \( B = \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} \). Their tensor products are \( A ⊗ B = \begin{bmatrix} a_1b_1 & a_1b_2 \\ a_2b_1 & a_2b_2 \end{bmatrix} \) and \( B ⊗ A = \begin{bmatrix} b_1a_1 & b_1a_2 \\ b_2a_1 & b_2a_2 \end{bmatrix} \). Comparing these matrices, we can see that they are not equal in general, showing that the tensor product is not commutative.

Step 1: Choose two vectors A and B

Let's choose two-dimensional column vectors A and B as follows: \( A = \begin{bmatrix} a_1 \\ a_2 \end{bmatrix} \), and \( B = \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} \)

Step 2: Calculate tensor product A ⊗ B

To calculate the tensor product of A ⊗ B, we need to take the outer product of A and B. The result will be a matrix where each element is the product of the corresponding elements in A and B: \( A ⊗ B = \begin{bmatrix} a_1 \\ a_2 \end{bmatrix} \otimes \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} = \begin{bmatrix} a_1 \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} \\ a_2 \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} a_1b_1 & a_1b_2 \\ a_2b_1 & a_2b_2 \end{bmatrix} \)

Step 3: Calculate tensor product B ⊗ A

Now, we need to compute the tensor product of B ⊗ A using the same approach: \( B ⊗ A = \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} \otimes \begin{bmatrix} a_1 \\ a_2 \end{bmatrix} = \begin{bmatrix} b_1 \begin{bmatrix} a_1 \\ a_2 \end{bmatrix} \\ b_2 \begin{bmatrix} a_1 \\ a_2 \end{bmatrix} \end{bmatrix} = \begin{bmatrix} b_1a_1 & b_1a_2 \\ b_2a_1 & b_2a_2 \end{bmatrix} \)

Step 4: Compare A ⊗ B and B ⊗ A

Now let's compare the two matrices we got as the results of the tensor products: \( A ⊗ B = \begin{bmatrix} a_1b_1 & a_1b_2 \\ a_2b_1 & a_2b_2 \end{bmatrix} \) and \( B ⊗ A = \begin{bmatrix} b_1a_1 & b_1a_2 \\ b_2a_1 & b_2a_2 \end{bmatrix} \) We can see that these two matrices are not, in general, equal to each other because the elements of A ⊗ B are different than the corresponding elements in B ⊗ A. Thus, we have shown that the tensor product is not commutative in general.

Key Concepts

Commutative Property

The commutative property is a fundamental concept from algebra that refers to the ability to change the order of two elements in an operation without affecting the outcome. In mathematical terms, for operations such as addition and multiplication, the commutative property holds, meaning:

• For addition: \( a + b = b + a \)
• For multiplication: \( a \, \cdot \, b = b \, \cdot \, a \)

However, this property does not apply to all operations. One notable exception is the tensor product of vectors or matrices. In the case of a tensor product, swapping the order of the operands usually results in a different matrix. For instance, if we have vectors \( A \) and \( B \), generally \( A \otimes B eq B \otimes A \). This distinction is a key point when performing tensor calculations.

Outer Product

To understand tensor products, it's crucial to first grasp the concept of the outer product. The outer product of two vectors \( A \) and \( B \), often denoted as \( A \otimes B \), forms a matrix. This matrix is constructed by multiplying each element of \( A \) with each element of \( B \). For example, consider: \( A = \begin{bmatrix} a_1 \ a_2 \end{bmatrix} \) and \( B = \begin{bmatrix} b_1 \ b_2 \end{bmatrix} \).
The outer product \( A \otimes B \) results in:

• \( \begin{bmatrix} a_1b_1 & a_1b_2 \ a_2b_1 & a_2b_2 \end{bmatrix} \)

The dimensions of the resulting matrix will be determined by the lengths of \( A \) and \( B \). In contrast to the dot product, which results in a scalar, the outer product forms a higher-dimensional structure, specifically a matrix. This process highlights the complexity of tensor operations and illustrates why the commutative property does not apply.

Matrix Operations

Matrix operations are essential parts of linear algebra, playing a crucial role in various areas such as physics, computer science, and engineering. Understanding how to perform operations with matrices, including addition, multiplication, and the tensor product, is vital.

• Matrix Addition
  This operation is straightforward, involving the addition of corresponding elements in matrices of the same size.
• Matrix Multiplication
  This is a more complex operation, which involves rows of the first matrix being multiplied by columns of the second matrix. Importantly, matrix multiplication is not commutative.
• Tensor Product
  As described, yields a matrix from two vectors or matrices by calculating the outer product. Unlike regular multiplication and addition, the order of matrices in a tensor product affects the result.

While performing these operations, it is important to pay attention to details like matrix dimensions and the order of operations. Mastering these skills is crucial for effectively solving more intricate problems involving matrices.