Matrix Multiplication
We will introduce matrix multiplication by first considering the special case of a matrix-vector product. In other words, the first matrix is and the second matrix is for some positive integers .
Matrix-Vector Multiplication
Let
We can now make a couple of observations about the matrix-vector product. The
first observation is part of the definition, but it is still worth pointing out. One way to understand the matrix-vector product is by thinking of it as a linear combination of the columns of , using the entries in as our coefficients.
We can also compute the product one entry at a time. First, let’s focus on the first row of . Next, let’s look a the second row of . Finally, let’s do the third row of .
In
order for the product to exist, and must have compatible dimensions. In
particular, vector must have as many components as the number of columns of
. (Otherwise, we would not be have a well-defined linear combination of
the columns.) So, if is an matrix, must be an vector. If we write these
dimensions next to each other, we will notice that the inner dimensions () must
match, while the outer dimensions, and , give us the dimensions of the
product.
If you are familiar with the dot product (see Dot Product and its Properties), you
may have noticed that each individual entry in the product matrix is the dot
product of a row of with . Thus, if the rows of are vectors , we can restate
Definition def:matrixvectormult as follows:
Let’s find another matrix-vector product.
Matrix-Matrix Multiplication
Matrix-matrix multiplication is simply an extension of the idea of matrix-vector multiplication. In order for the product definition to work, matrix dimensions must be compatible. Let be an matrix, and let be an matrix, then the product will be an matrix. Just like with vector products, the inner dimensions must be the same, while the outer dimensions, and , give us the dimensions of the product.
Let be an matrix whose rows are vectors , . Let be an matrix with columns
. Then the entries of the matrix product are given by the dot products
So the -entry of is the dot product of the row of and the column of .
In terms of components, if the row of is and the column of is then the -entry of is given by
It is possible to use linear combinations rather than dot products to compute a
matrix-matrix product. If the columns of matrix are given by , then the matrix
product consists of columns, each of which is a matrix-vector product:
If we return to Example ex:matmatproduct above, we may use this approach to compute and
Suppose that we know that satisfies
Use this information to express as a linear combination of the columns of .
Properties of Matrix Multiplication
Let
Observe that both and are defined, and both products are matrices. Let’s compute
the two products
Clearly . We say that and do not commute.
While it is possible to find specific matrices that commute, matrix multiplication is
not commutative in general.
One example of an square matrix that commutes with all matrices is the matrix defined by has 1’s along the main diagonal and 0’s everywhere else. It is often useful to think of as a matrix whose column (and row) is , the standard unit vector of . When the dimensions of are clear from the context, or irrelevant, we will omit the subscript and simply refer to this matrix as .
You can easily convince yourself that commutes with all square matrices of appropriate dimensions. Let Verify that and .
Because acts like the multiplicative identity in regular multiplication, (or ) is called the identity matrix.
Next we list several important properties of matrix multiplication. These properties hold only when matrix sizes are such that the products are defined.
- Proof
- We prove item:matrixproperties1 using the expression in (eq:ijentrymatrixproduct) for the -entry of a matrix product.
(The proof of item:matrixproperties2 is similar.) The -entry of is given by
We recognize the right hand side as the -entry of . Thus .
For item:matrixproperties4, For item:identitymatrix, the -entry of the product is given by the dot product of the row of with the standard unit vector . Clearly, this dot product is . Because the -entry of the product is equal to the -entry , we conclude that . The proof that is similar.
Note that we skipped the proof of item:matrixproperties3, which is quite cumbersome using sigma notation. We will easily tackle this proof later in the course when we cover Composition of Linear Transformations.
Practice Problems
Problems prob:matproddim1-prob:matproddim2 Predict the dimensions of each product.
Problems prob:matprod1-prob:matprod3 Find each product.
Text Source
The section on Properties of Matrix Multiplication is an adaptation of Section 2.1 of Ken Kuttler’s A First Course in Linear Algebra. (CC-BY)
Ken Kuttler, A First Course in Linear Algebra, Lyryx 2017, Open Edition, p. 67-68.