We introduce matrix-vector and matrix-matrix multiplication, and interpret matrix-vector multiplication as linear combination of the columns of the matrix.

MAT-0020: Matrix Multiplication

We will introduce matrix multiplication by first considering the spacial case of a matrix-vector product.

Matrix-Vector Multiplication

The following example illustrates the pattern of matrix-vector multiplication. We can now make a couple of observations about the matrix-vector product. The first observation is part of the definition, but it is still worth pointing out.

Let’s find another matrix-vector product.

Matrix-Matrix Multiplication

Matrix-matrix multiplication is simply an extension of the idea of matrix-vector multiplication. In order for the product definition to work, matrix dimensions must be compatible. Let be an matrix, and let be an matrix, then the product will be an matrix. Just like with vector products, the inner dimensions must be the same, while the outer dimensions, and , give us the dimensions of the product.

It is worth pointing out that the -entry of is the dot product of the row of and the column of .

In terms of components, if the row of is and the column of is then the -entry of is given by

Properties of Matrix Multiplication

Let Observe that both and are defined, and both products are matrices. Let’s compute the two products Clearly . We say that and do not commute.
While it is possible to find specific matrices that commute, matrix multiplication is not commutative in general.

One example of an square matrix that commutes with all matrices is the matrix defined by has 1’s along the main diagonal and 0’s everywhere else. It is often useful to think of as a matrix whose column (and row) is , the standard unit vector of . When the dimensions of are clear from the context, or irrelevant, we will omit the subscript and simply refer to this matrix as .

You can easily convince yourself that commutes with all square matrices of appropriate dimensions. Let Verify that and .

Because acts like the multiplicative identity in regular multiplication, (or ) is called the identity matrix.

Next we list several important properties of matrix multiplication. These properties hold only when matrix sizes are such that the products are defined.

We will prove Properties item:matrixproperties1 and item:identitymatrix. The remaining properties are left to the reader. See Practice Problems prob:matrixproperties2 and prob:matrixproperties3.

Proof of Property item:matrixproperties1:
We will prove this statement using expression in (eq:ijentrymatrixproduct) for the -entry of a matrix product. The -entry of is given by

We recognize the right hand side as the -entry of . Thus .

Proof of Property item:identitymatrix:
The -entry of the product is given by the dot product of the row of with the standard unit vector . Clearly, this dot product is . Because the -entry of the product is equal to the -entry , we conclude that . The proof that is similar.

Matrix-Vector Product as a Linear Combination of the Columns of the Matrix

Consider the product: We can rewrite this product as follows:

Thus, we have expressed as a linear combination of the columns of .

The following formula generalizes our observations in Exploration init:prodaslincomb.

Practice Problems

Explain why the following product is not defined.
Predict the dimensions of each product.
Dimensions of product:
Dimensions of product:
Find each product.
Prove Property item:matrixproperties3 of Theorem th:propertiesofmatrixmultiplication. This proof can be done by comparing the -entries on both sides of the equal sign. (If you find that the notation is a nightmare, an alternative proof will be presented later in Theorem th:associativematrixmult.)
Express the given product as a linear combination of the columns of the matrix.

Text Source

The section on Properties of Matrix Multiplication is an adaptation of Section 2.1 of Ken Kuttler’s A First Course in Linear Algebra. (CC-BY)

Ken Kuttler, A First Course in Linear Algebra, Lyryx 2017, Open Edition, p. 67-68.