You are about to erase your work on this activity. Are you sure you want to do this?
Updated Version Available
There is an updated version of this activity. If you update to the most recent version of this activity, then your current progress on this activity will be erased. Regardless, your record of completion will remain. How would you like to proceed?
Mathematical Expression Editor
Diagonalizable Matrices and Multiplicity
Recall that a diagonal matrix\(D\) is a matrix containing a zero in every entry except
those on the main diagonal. More precisely, if \(d_{ij}\) is the \(ij^{th}\) entry of a diagonal matrix \(D\),
then \(d_{ij}=0\) unless \(i=j\). Such matrices look like the following.
Diagonal matrices have some nice properties, as we demonstrate below.
Let \(M =\begin{bmatrix}1 & 2 & 3\\ 4&5&6\\7&8&9\end{bmatrix}\) and let \(D =\begin{bmatrix}2 & 0 & 0\\ 0&-5&0\\0&0&10\end{bmatrix}\). Compute \(MD\) and \(DM\) using Octave.
% Define M
M=[1 2 3; 4 5 6; 7 8 9]
% The function diag(vector) creates a square matrix with diagonal
%vector and the remaining entries zero.
D=diag([2 -5 10])
% The function diag() can also be used to return the diagonal of a matrix.
diag_M=diag(M)
% Understanding multiplication by a diagonal matrix
product_MD=M*D
product_DM=D*M
% Raising a diagonal matrix to a power
D5=D^5
Notice the patterns present in the product matrices.
Each row of \(DM\) is the same as its corresponding row of \(M\) multiplied by the
scalar which is the corresponding diagonal element of \(D\).
In the product \(MD\), it is the columns of \(M\) that have been multiplied by the
diagonal elements.
Raising \(D\) to the \(5^{th}\) power amounts to raising each diagonal element to the \(5^{th}\)
power.
These patterns hold in general for any diagonal matrix, and they are fundamental to
understanding diagonalization and its uses. We discuss diagonalization next.
Let \(A\) be an \(n\times n\) matrix. Then \(A\) is said to be diagonalizable if there exists an invertible
matrix \(P\) such that
\begin{equation*} P^{-1}AP=D \end{equation*}
where \(D\) is a diagonal matrix. In other words, a matrix \(A\) is
diagonalizable if it is similar to a diagonal matrix, \(A \sim D\).
If we are given a matrix \(A\) that is diagonalizable, then we can write \(P^{-1}AP=D\) for some matrix \(P\),
or, equivalently,
If we pause to examine Equation (eq:understand_diag), the work that we did in
Exploration init:multiplydiag can help us to understand how to find \(P\) that will diagonalize \(A\). The
product \(PD\) is formed by multiplying each column of \(P\) by a scalar which is the
corresponding element on the diagonal of \(D\). To restate this, if \(\vec {x}_i\) is column \(i\) in our
matrix \(P\), then Equation (eq:understand_diag) tells us that
where \(\lambda _i\) is the \(i\)th diagonal element of
\(D\).
Of course, Equation (eq:ev_ew_diag) is very familiar! We see that if we are able to diagonalize a
matrix \(A\), the columns of matrix \(P\) will be the eigenvectors of \(A\), and the corresponding
diagonal entries of \(D\) will be the corresponding eigenvalues of \(A\). This is summed up in
the following theorem.
An \(n\times n\) matrix \(A\) is diagonalizable if and only if there is an invertible matrix \(P\) given
by
showing the \(\vec {x}_i\) are eigenvectors of \(A\) and the \(\lambda _{i}\) are eigenvalues.
Notice that because the matrix \(P\) defined above is invertible it follows that the set of
eigenvectors of \(A\), \(\left \{ \vec {x}_1 , \vec {x}_2 , \ldots , , \vec {x}_n \right \}\), is a basis of \(\mathbb {R}^n\).
We demonstrate the concept given in the above theorem in the next example. Note
that not only are the columns of the matrix \(P\) formed by eigenvectors, but \(P\) must
be invertible, and therefore must consist of a linearly independent set of
eigenvectors.
Find an invertible matrix \(P\) and a diagonal matrix \(D\) such that \(P^{-1}AP=D\).
We will use eigenvectors of \(A\) as the columns of \(P\), and the corresponding eigenvalues of \(A\)
as the diagonal entries of \(D\). The eigenvalues of \(A\) are \(\lambda _1 =2,\lambda _2 = 2\), and \(\lambda _3 = 6\). We leave these
computations as exercises, as well as the computations to find a basis for each
eigenspace.
One possible basis for \(\mathcal {S}_2\), the eigenspace corresponding to \(2\), is \(\left \{ \begin{bmatrix} -2 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix} \right \}\), while a basis for \(\mathcal {S}_6\) is
given by \(\left \{\begin{bmatrix} 0 \\ 1 \\ -2 \end{bmatrix}\right \}\).
We construct the matrix \(P\) by using these basis elements as columns.
You can see that the result here is a diagonal matrix where the entries on the
main diagonal are the eigenvalues of \(A\). Notice that eigenvalues on the main
diagonal must be in the same order as the corresponding eigenvectors in \(P\).
We will continue to use matrices from Example ex:diagonalizematrix to illustrate why diagonalizable
matrices are easier to work with. Let \(A=\begin{bmatrix} 2 & 0 & 0 \\ 1 & 4 & -1 \\ -2 & -4 & 4 \end{bmatrix}\) and let \(D=\begin{bmatrix} 2 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 6 \end{bmatrix}\). Would it be easier to compute \(A^5\) or \(D^5\) if
you had to do so by hand, without a computer? Certainly \(D^5\) is easier, due to the
number of zero entries!
Raising a diagonal matrix to a power amounts to simply raising each entry to that
same power, whereas computing \(A^5\) requires many more calculations. However, we
learned in Example ex:diagonalizematrix that \(A\) is similar to \(D\). We will use this to make our computation
easier as follows
The savings in work would certainly be more pronounced for larger matrices or for
powers larger that 5.
In Exploration exp:motivate_diagonalization, because matrix \(A\) was diagonalizable, we were able to cut
down on computations. When we chose to work with \(D\) and \(P\) instead of \(A\) we
worked with the eigenvalues and eigenvectors of \(A\). Each column of \(P\) is an
eigenvector of \(A\), and so we repeatedly made use of the following theorem (with
\(m=5\)).
Let \(A\) be an \(n \times n\) matrix and suppose \(A\vec {x}=\lambda \vec {x}\). Then \(A^m \vec {x} = \lambda ^m \vec {x}\)
We prove this theorem by induction on \(m\). Clearly \(A^m \vec {x} = \lambda ^m \vec {x}\) holds when \(m=1\), as that was given. For
the inductive step, suppose that we know \(A^{m-1} \vec {x} = \lambda ^{m-1} \vec {x}\). Then
Matrix \(A\) from the Example ex:diagonalizematrix and Exploration exp:motivate_diagonalization had a repeated eigenvalue of 2. The
next theorem and corollary show that matrices which have distinct eigenvalues
(where none are repeated) have desirable properties.
Let \(A\) be an \(n\times n\) matrix, and suppose that \(A\) has distinct eigenvalues \(\lambda _1, \lambda _2, \ldots , \lambda _m\). For each \(i\), let \(\vec {x}_i\) be a
\(\lambda _i\)-eigenvector of \(A\). Then \(\{ \vec {x}_1, \vec {x}_2, \ldots , \vec {x}_m\}\) is linearly independent.
We prove this by induction on \(m\), the number of vectors in the set. If \(m = 1\), then \(\{\vec {x}_{1}\}\) is a
linearly independent set because \(\vec {x}_{1} \neq \vec {0}\). In general, suppose we have established that the
theorem is true for some \(m \geq 1\). Given eigenvectors \(\{\vec {x}_{1}, \vec {x}_{2}, \dots , \vec {x}_{m+1}\}\), suppose
We must show that each \(c_{i} = 0\).
Multiply both sides of (eq:thm_proof_5_5_4_1) on the left by \(A\) and use the fact that \(A\vec {x}_{i} = \lambda _{i}\vec {x}_{i}\) to get
and so \(c_{2} = c_{3} = \dots = c_{m+1} = 0\) because the \(\lambda _{i}\) are distinct. It follows
that (eq:thm_proof_5_5_4_1) becomes \(c_{1}\vec {x}_{1} = \vec {0}\), which implies that \(c_{1} = 0\) because \(\vec {x}_{1} \neq \vec {0}\), and the proof is complete.
The corollary that follows from this theorem gives a useful tool in determining if \(A\) is
diagonalizable.
Let \(A\) be an \(n \times n\) matrix and suppose it has \(n\) distinct eigenvalues. Then it follows that \(A\) is
diagonalizable.
Note that Corollary th:distincteigenvalues is NOT an “if and only if statement". This means that if \(A\) has
repeated eigenvalues it is still sometimes possible to diagonalize \(A\), as seen in Example ex:diagonalizematrix.
If we are able to diagonalize \(A\), say \(A=PDP^{-1}\), we say that \(PDP^{-1}\) is an eigenvalue decomposition of \(A\).
Not every matrix has an eigenvalue decomposition. Sometimes we cannot find an
invertible matrix \(P\) such that \(P^{-1}AP=D\). Consider the following example.
If possible, find an invertible matrix \(P\) and a diagonal matrix \(D\) so that
\(P^{-1}AP=D\).
We see immediately (how?) that the eigenvalues of \(A\) are \(\lambda _1 =1\) and \(\lambda _2=1\). To find \(P\), the next step
would be to find a basis for the corresponding eigenspace \(\mathcal {S}_1\). We solve the
equation \(\left ( A - \lambda I \right ) \vec {x} = \vec {0}\). Writing this equation as an augmented matrix, we already have a
matrix in row echelon form:
so a basis for the eigenspace \(\mathcal {S}_1\) is given by \(\begin{bmatrix} 1 \\ 0 \end{bmatrix}\). It is easy to see that we cannot form an
invertible matrix \(P\), because any two eigenvectors will be of the form \(\begin{bmatrix} t \\ 0 \end{bmatrix}\), and so the
second row of \(P\) would have a row of zeros, and \(P\) could not be invertible. Hence \(A\) cannot
be diagonalized.
We saw earlier in Corollary th:distincteigenvalues that an \(n \times n\) matrix with \(n\) distinct
eigenvalues is diagonalizable. It turns out that there are other useful diagonalizability
tests.
Recall that the algebraic multiplicity of an eigenvalue \(\lambda \) is the number of times that it
occurs as a root of the characteristic polynomial.
The geometric multiplicity of an eigenvalue \(\lambda \) is the dimension of the corresponding
eigenspace \(\mathcal {S}_\lambda \).
Consider now the following lemma.
Let \(A\) be an \(n\times n\) matrix, and let \(\mathcal {S}_{\lambda _1}\) be the eigenspace corresponding to the eigenvalue \(\lambda _1\) which
has algebraic multiplicity \(m\). Then
\[\mbox {dim}(\mathcal {S}_{\lambda _1})\leq m\]
In other words, the geometric multiplicity of an eigenvalue is less than or equal to the
algebraic multiplicity of that same eigenvalue.
Let \(k\) be the geometric multiplicity of \(\lambda _1\), i.e., \(k=\mbox {dim}(\mathcal {S}_{\lambda _1})\). Suppose \(\left \{\vec {x}_1, \vec {x}_2, \ldots ,\vec {x}_k\right \}\) is a basis for the eigenspace \(\mathcal {S}_{\lambda _1}\).
Let \(P\) be any invertible matrix having \(\vec {x}_1, \vec {x}_2, \ldots ,\vec {x}_k\) as its first \(k\) columns, say
\[P=\begin{bmatrix} B&C \end{bmatrix} \quad \text {and} \quad P^{-1}=\begin{bmatrix} D \\ E \end{bmatrix}\]
where \(B\) is \(n \times k\), \(C\) is \(n \times (n-k)\), \(D\) is \(k \times n\), and \(E\) is \((n-k) \times n\). We observe \(I_n = P^{-1}P = \left [\begin{array}{c|c} DB & DC \\ \hline EB & EC \end{array}\right ] \). This implies
We finish the proof by comparing the characteristic polynomials on both
sides of this equation, and making use of the fact that similar matrices have the same
characteristic polynomials.
We see that the characteristic polynomial of \(A\) has \((\lambda _1 - \lambda )^k\) as a factor. This tells us
that algebraic multiplicity of \(\lambda _1\) is at least \(k\), proving the desired inequality.
This result tells us that if \(\lambda \) is an eigenvalue of \(A\), then the number of linearly
independent \(\lambda \)-eigenvectors is never more than the multiplicity of \(\lambda \). We now use this
fact to provide a useful diagonalizability condition.
Let \(A\) be an \(n \times n\) matrix \(A\). Then \(A\) is diagonalizable if and only if for each eigenvalue \(\lambda \)
of \(A\), the algebraic multiplicity of \(\lambda \) is equal to the geometric multiplicity of \(\lambda \).
Suppose \(A\) is diagonalizable and let \(\lambda _1, \ldots , \lambda _t\) be the distinct eigenvalues of \(A\), with algebraic
multiplicities \(m_1, \ldots , m_t\), respectively and geometric multiplicities \(k_1, \ldots , k_t\), respectively. Since \(A\) is
diagonalizable, Theorem th:eigenvectorsanddiagonalizable implies that \( k_1+\cdots +k_t=n\). By applying Lemma lemma:dimeigenspace\(t\) times, we
have
\[n = k_1+\cdots +k_t \le m_1+\cdots +m_t = n,\]
which is only possible if \(k_i=m_i\) for \(i=1,\ldots ,t\).
Conversely, if the geometric multiplicity equals the algebraic multiplicity of each
eigenvalue, then obtaining a basis for each eigenspace yields \(n\) eigenvectors. Applying
Theorem th:linindepeigenvectors, we know that these \(n\) eigenvectors are linearly independent, so Theorem th:eigenvectorsanddiagonalizable
implies that \(A\) is diagonalizable.
Geometric multiplicity of \(\lambda _1\) is \(\answer {1}\).
Explain why \(A\) is not diagonalizable.
Let \(A\) denote an \(n \times n\) upper triangular matrix. If all the main diagonal entries of \(A\) are
distinct, show that \(A\) is diagonalizable.
Let \(A\) be a diagonalizable \(n \times n\) matrix with eigenvalues \(\lambda _{1}, \lambda _{2}, \dots , \lambda _{n}\) (including multiplicities). Show
that:
(a)
\(\det A = \lambda _{1}\lambda _{2}\cdots \lambda _{n}\)