We introduce matrix-vector and matrix-matrix multiplication, and interpret matrix-vector multiplication as linear combination of the columns of the matrix.
MAT-0020: Matrix Multiplication
We will introduce matrix multiplication by first considering the spacial case of a matrix-vector product.
Matrix-Vector Multiplication
The following example illustrates the pattern of matrix-vector multiplication. We can now make a couple of observations about the matrix-vector product. The first observation is part of the definition, but it is still worth pointing out.In order
for the product to exist, and must have compatible dimensions. In particular,
vector must have as many components as the number of columns of . So, if is an
matrix, must be an vector. If we write these dimensions next to each other, we will
notice that the inner dimensions () must match, while the outer dimensions, and ,
give us the dimensions of the product.
If you are familiar with the dot product, you may have noticed that the entries in the
product matrix are dot products of the rows of with . Thus, if the rows of are
vectors , we can restate Definition def:matrixvectormult as follows:
Let’s find another matrix-vector product.
Matrix-Matrix Multiplication
Matrix-matrix multiplication is simply an extension of the idea of matrix-vector multiplication. In order for the product definition to work, matrix dimensions must be compatible. Let be an matrix, and let be an matrix, then the product will be an matrix. Just like with vector products, the inner dimensions must be the same, while the outer dimensions, and , give us the dimensions of the product.
Let be an matrix whose rows are vectors , . Let be
an matrix with columns . Then the product is given by
It is worth pointing out that the -entry of is the dot product of the row of and the column of .
In terms of components, if the row of is and the column of is then the -entry of is given by
Properties of Matrix Multiplication
Let
Observe that both and are defined, and both products are matrices. Let’s compute
the two products
Clearly . We say that and do not commute.
While it is possible to find specific matrices that commute, matrix multiplication is
not commutative in general.
One example of an square matrix that commutes with all matrices is the matrix defined by has 1’s along the main diagonal and 0’s everywhere else. It is often useful to think of as a matrix whose column (and row) is , the standard unit vector of . When the dimensions of are clear from the context, or irrelevant, we will omit the subscript and simply refer to this matrix as .
You can easily convince yourself that commutes with all square matrices of appropriate dimensions. Let Verify that and .
Because acts like the multiplicative identity in regular multiplication, (or ) is called the identity matrix.
Next we list several important properties of matrix multiplication. These properties hold only when matrix sizes are such that the products are defined.
We will prove Properties item:matrixproperties1 and item:identitymatrix. The remaining properties are left to the reader. See Practice Problems prob:matrixproperties2 and prob:matrixproperties3.
- Proof of Property item:matrixproperties1:
- We will prove this statement using expression in (eq:ijentrymatrixproduct)
for the -entry of a matrix product. The -entry of is given by
We recognize the right hand side as the -entry of . Thus .
- Proof of Property item:identitymatrix:
- The -entry of the product is given by the dot product of the row of with the standard unit vector . Clearly, this dot product is . Because the -entry of the product is equal to the -entry , we conclude that . The proof that is similar.
Matrix-Vector Product as a Linear Combination of the Columns of the Matrix
Consider the product:
We can rewrite this product as follows:
Thus, we have expressed as a linear combination of the columns of .
The following formula generalizes our observations in Exploration init:prodaslincomb.
Suppose that we know that satisfies
Use this information to express as a linear combination of the columns of .
Practice Problems
Prove Properties item:matrixproperties2 and item:matrixproperties4 of Theorem th:propertiesofmatrixmultiplication.
Prove Property item:matrixproperties3 of Theorem th:propertiesofmatrixmultiplication. This proof can be done by comparing the -entries on
both sides of the equal sign. (If you find that the notation is a nightmare, an
alternative proof will be presented later in Theorem th:associativematrixmult.)
Text Source
The section on Properties of Matrix Multiplication is an adaptation of Section 2.1 of Ken Kuttler’s A First Course in Linear Algebra. (CC-BY)
Ken Kuttler, A First Course in Linear Algebra, Lyryx 2017, Open Edition, p. 67-68.