You are about to erase your work on this activity. Are you sure you want to do this?
Updated Version Available
There is an updated version of this activity. If you update to the most recent version of this activity, then your current progress on this activity will be erased. Regardless, your record of completion will remain. How would you like to proceed?
Mathematical Expression Editor
Orthogonal Matrices and Symmetric Matrices
Recall that an \(n \times n\) matrix \(A\) is diagonalizable if and only if it has \(n\) linearly independent
eigenvectors. (see Diagonalizable Matrices and Multiplicity) Moreover, the
matrix \(P\) with these eigenvectors as columns is a diagonalizing matrix for \(A\),
that is
\begin{equation*} P^{-1}AP \mbox { is diagonal.} \end{equation*}
As we have seen, the nice bases of \(\RR ^n\) are the orthogonal ones, so a
natural question is: which \(n \times n\) matrices have \(n\) orthogonal eigenvectors, so that
columns of \(P\) form an orthogonal basis for \(\RR ^n\)? These turn out to be precisely the
symmetric matrices (matrices for which \(A=A^T\)), and this is the main result of this
section.
Orthogonal Matrices
A collection of non-zero, pairwise orthogonal vectors in \(\RR ^n\) is called an orthogonal set of
vectors. An orthogonal set of vectors is called orthonormal if \(\norm {\vec {q}} = 1\) for each vector \(\vec {q}\) in the
set. A set of orthogonal vectors \(\{\vec {v}_{1}, \vec {v}_{2}, \dots , \vec {v}_{k}\}\) can be “normalized”, i.e. converted into an
orthonormal set \(\left \{ \frac {1}{\norm {\vec {v}_{1}}}\vec {v}_{1}, \frac {1}{\norm {\vec {v}_{2}}}\vec {v}_{2}, \dots , \frac {1}{\norm {\vec {v}_{k}}}\vec {v}_{k} \right \}\). In particular, if a matrix \(A\) has \(n\) orthogonal eigenvectors, they can
(by normalizing) be taken to be orthonormal. The corresponding diagonalizing matrix
(we will use \(Q\) instead of \(P\)) has orthonormal columns, and such matrices are very easy
to invert.
The following conditions are equivalent for an \(n \times n\) matrix \(Q\).
(a)
\(Q\) is invertible and \(Q^{-1} = Q^{T}\).
(b)
The rows of \(Q\) are orthonormal.
(c)
The columns of \(Q\) are orthonormal.
First note that condition (a) is equivalent to \(Q^{T}Q = I\). Let \(\vec {q}_{1}, \vec {q}_{2}, \dots , \vec {q}_{n}\) denote the columns of \(Q\). Then \(\vec {q}_{i}^{T}\) is
the \(i\)th row of \(Q^{T}\), so the \((i, j)\)-entry of \(Q^{T}Q\) is \(\vec {q}_{i} \dotp \vec {q}_{j}\). Thus \(Q^{T}Q = I\) means that \(\vec {q}_{i} \dotp \vec {q}_{j} = 0\) if \(i \neq j\) and \(\vec {q}_{i} \dotp \vec {q}_{j} = 1\) if \(i = j\). Hence condition (a)
is equivalent to (c). The proof of the equivalence of (a) and (b) is similar.
Orthogonal Matrices An \(n \times n\) matrix \(Q\) is called an orthogonal matrix if it satisfies one
(and hence all) of the conditions in Theorem 1.
The rotation matrix \(\begin{bmatrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{bmatrix}\) is orthogonal for any angle \(\theta \).
See Practice Problem ??.
In view of (b) and (c) of Theorem 1, orthonormal matrix might be a better name.
But orthogonal matrix is standard.
It is not enough that the rows of a matrix \(A\) are merely orthogonal for \(A\) to be an
orthogonal matrix. Here is an example.
Let \(A=\begin{bmatrix} 2&1&1\\-1&1&1\\0&-1&1 \end{bmatrix}\)
(a)
Check that matrix \(A\) has rows that are orthogonal.
(b)
Check that matrix \(A\) has columns that are NOT orthogonal.
(c)
Check that matrix \(A\) has rows that are NOT orthonormal.
(d)
Create a matrix \(Q\) by normalizing each of the rows of \(A\).
(e)
Check that \(Q\) is an orthogonal matrix.
Click the arrow to see the answer.
You should get \(Q = \begin{bmatrix} \frac {2}{\sqrt {6}} & \frac {1}{\sqrt {6}} & \frac {1}{\sqrt {6}} \\ \frac {-1}{\sqrt {3}} & \frac {1}{\sqrt {3}} & \frac {1}{\sqrt {3}} \\ 0 & \frac {-1}{\sqrt {2}} & \frac {1}{\sqrt {2}} \end{bmatrix}\), and one can check that this is
orthogonal in a number of ways.
This exploration can certainly be done by hand (although it takes some time), but it
also makes for a very nice Octave exercise.
%Exploration from Section 9.4 Orthogonal Matrices and Symmetric Matrices
A=[2 1 1; -1 1 1; 0 -1 1]
%Check that matrix A has rows that are orthogonal.
A(1,:)*transpose(A(2,:))
A(2,:)*transpose(A(3,:))
A(1,:)*transpose(A(3,:))
%Check that matrix A has columns that are NOT orthogonal.
transpose(A(:,1))*A(:,2)
%(This is 1 of 3 calculations to do.)
%Check that matrix A in the Octave window has rows that are NOT orthonormal.
%(See the results from the first question.)
%Create a matrix Q by normalizing each of the rows of A.
q1=A(1,:)/norm(A(1,:));
q2=A(2,:)/norm(A(2,:));
q3=A(3,:)/norm(A(3,:));
Q = [q1;q2;q3]
%Check that Q is an orthogonal matrix.
Q*transpose(Q)
%(You may get numbers close to zero in some places you expect
%to get zero due to rounding error)
We studied the idea of closure when we studied Subspaces of \(\RR ^n\). The next theorem
tells us that orthogonal matrices are closed under matrix multiplication.
(a)
If \(P\) and \(Q\) are orthogonal matrices, then \(PQ\) is also orthogonal. (We say that
the set of orthogonal matices is closed under matrix multiplication.)
(b)
If \(P\) is an orthogonal matrix, then so is \(P^{-1} = P^{T}\).
For (a), \(P\) and \(Q\) are invertible, so \(PQ\) is also invertible and
\begin{equation*} (P^{-1})^{-1} = P = (P^{T})^{T} = (P^{-1})^{T} \end{equation*}
shows that \(P^{-1}\) is orthogonal.
Symmetric Matrices
We now shift our focus from orthogonal matrices to another important class of \(n \times n\)
matrices called symmetric matrices. A symmetric matrix is a matrix which is equal
to its transpose. We saw a few examples of such matrices in Transpose of a
Matrix.
When we began our study of eigenvalues and eigenvectors, we saw numerous
examples of matrices with entries that were real numbers with eigenvalues
that were complex numbers. It can be shown that symmetric matrices only
have real eigenvalues. We also learned that some matrices are diagonalizable
while other matrices are not. It turns out that every symmetric matrix is
diagonalizable. In fact, we can say more, but first we need the following
definition.
An \(n \times n\) matrix \(A\) is said to be orthogonally diagonalizable if an orthogonal matrix \(Q\) can be
found such that \(Q^{-1}AQ = Q^{T}AQ\) is diagonal.
We have learned earlier that when we diagonalize a matrix \(A\), we write \(P^{-1}AP=D\) for some
matrix \(P\) where \(D\) is diagonal, and the diagonal entries are the eigenvalues of \(A\). We
have also learned that the columns of the matrix \(P\) are the corresponding
eigenvectors of \(A\). So when a matrix is orthogonally diagonalizable, we are able to
accomplish the diagonalization using a matrix \(Q\) consisting of \(n\) eigenvectors
that form an orthonormal basis for \(\RR ^n\). The following remarkable theorem
shows that the matrices that have this property are precisely the symmetric
matrices.
Real Spectral Theorem Let \(A\) be an \(n \times n\) matrix. Then \(A\) is symmetric if and only if \(A\) is
orthogonally diagonalizable.
If \(A\) is orthogonally diagonalizable, then it is an easy exercise to prove that it is
symmetric. You are asked to do this in Practice Problem 27.
To prove the “only if” part of this theorem, we assume \(A\) is symmetric, and we need to
show it is orthogonally diagonalizable. We proceed by induction on \(n\), the size of the
symmetric matrix. If \(n = 1\), \(A\) is already diagonal. If \(n > 1\), assume that we know the “only if”
statement holds for \((n - 1) \times (n - 1)\) symmetric matrices. Let \(\lambda _{1}\) be an eigenvalue of \(A\), and
let \(A\vec {x}_{1} = \lambda _{1}\vec {x}_{1}\), where \(\norm {\vec {x}_{1}} = 1\). Next, set \(\vec {q}_{1}=\vec {x}_{1}\), and use the Gram-Schmidt algorithm to find an
orthonormal basis \(\{\vec {q}_{1}, \vec {q}_{2}, \dots , \vec {q}_{n}\}\) for \(\RR ^n\). Let \(Q_{1} = \begin{bmatrix} | & | & & | \\ \vec {q}_1 & \vec {q}_2 & \cdots & \vec {q}_n \\ | & | & & | \end{bmatrix}\), so that \(Q_{1}\) is an orthogonal matrix. We have
where the block \(B\) has dimensions \(1 \times (n-1)\), and the block under \(\lambda _1\) is a \((n-1) \times 1\) zero matrix, because of
the orthogonality of the basis vectors.
Next, using the fact that \(A\) is symmetric, we notice that
so \(Q_{1}^TAQ_{1}\) is symmetric. It follows that \(B\) is also a zero matrix and that \(A_{1}\) is symmetric. Since \(A_{1}\)
is an \((n - 1) \times (n - 1)\) symmetric matrix, we may apply the inductive hypothesis, so there exists an \((n - 1) \times (n - 1)\)
orthogonal matrix \(Q\) such that \(Q^{T}A_{1}Q = D_{1}\) is diagonal. We observe that \(Q_{2} = \begin{bmatrix} 1 & 0\\ 0 & Q \end{bmatrix}\) is orthogonal, and we
compute:
is diagonal. Because \(Q_{1}Q_{2}\) is orthogonal by Theorem 7(a), this completes the proof.
Because the eigenvalues of a real symmetric matrix are real, Theorem 10 is called the
Real Spectral Theorem, and the set of distinct eigenvalues is called the spectrum of
the matrix. A similar result holds for matrices with complex entries (Theorem
??
).
Find an orthogonal matrix \(Q\) such that \(Q^{-1}AQ\) is diagonal, where \(A = \begin{bmatrix} 1 & 0 & -1 \\ 0 & 1 & 2 \\ -1 & 2 & 5 \end{bmatrix}\).
respectively. Moreover, by what at first appears to be
remarkably good luck, these eigenvectors are orthogonal. We have \(\norm {\vec {x}_{1}}^{2} = 6\), \(\norm {\vec {x}_{2}}^{2} = 5\), and \(\norm {\vec {x}_{3}}^{2} = 30\), so
Actually, the fact that the eigenvectors in Example 12 are orthogonal is no
coincidence. These vectors certainly must be linearly independent (they correspond
to distinct eigenvalues). We will see that the fact that the matrix is symmetric
implies that the eigenvectors are orthogonal. To prove this we need the following
useful fact about symmetric matrices.
Hence \((\lambda - \mu )(\vec {x} \dotp \vec {y}) = 0\), and so \(\vec {x} \dotp \vec {y} = 0\) because \(\lambda \neq \mu \).
Now the procedure for diagonalizing a symmetric \(n \times n\) matrix is clear. Find the distinct
eigenvalues and find orthonormal bases for each eigenspace (the Gram-Schmidt
algorithm may be needed when there is a repeated eigenvalue). Then the set of all
these basis vectors is orthonormal (by Theorem 17) and contains \(n\) vectors. Here is an
example.
Hence the distinct eigenvalues are \(0\) and \(9\) are of
algebraic multiplicity \(1\) and \(2\), respectively. The geometric multiplicities must
be the same, for \(A\) is diagonalizable, being symmetric. It follows that \(\mbox {dim}(\mathcal {S}_0) = 1\) and \(\mbox {dim}(\mathcal {S}_9) = 2\).
Gaussian elimination gives
The eigenvectors in \(\mathcal {S}_{9}\) are both orthogonal to \(\vec {x}_{1}\) as
Theorem 17 guarantees, but not to each other. However, an orthogonal basis
can be found using projections. (See Gram-Schmidt Orthogonalization)
is an orthogonal matrix such that \(Q^{-1}AQ\) is
diagonal.
It is worth noting that other, more convenient, diagonalizing matrices \(Q\) exist. For
example, \(\vec {y}_{2} = \begin{bmatrix} 2 \\ 1 \\ 2 \end{bmatrix}\) and \(\vec {y}_{3} = \begin{bmatrix} -2 \\ 2 \\ 1 \end{bmatrix}\) lie in \(\mathcal {S}_{9}(A)\) and they are orthogonal. Moreover, they both have norm \(3\) (as
does \(\vec {x}_{1}\)), so
is a nicer orthogonal matrix with the property that \(\hat {Q}^{-1}A\hat {Q}\) is diagonal.
Let \(A\) be an \(n \times n\) matrix. \(A\) has an orthonormal set of \(n\) eigenvectors if and only if \(A\) is
orthogonally diagonalizable.
Let \(\vec {q}_{1}, \vec {q}_{2}, \dots , \vec {q}_{n}\) be orthonormal eigenvectors of \(A\) with corresponding eigenvalues \(\lambda _1, \lambda _2, \ldots , \lambda _n\) . We
must show \(A\) is orthogonally diagonalizable. Let \(Q = \begin{bmatrix} | & | & & | \\ \vec {q}_1 & \vec {q}_2 & \cdots & \vec {q}_n \\ | & | & & | \end{bmatrix}\) so that \(Q\) is orthogonal. We
have
where \(D\) is the diagonal matrix with diagonal entries \(\lambda _1, \lambda _2, \ldots , \lambda _n\). But then \(Q^TAQ=D\), proving this half of
the theorem.
For the converse, if \(A\) is orthogonally diagonalizable, then by Theorem 10 it is
symmetric. But then Theorem 17 tells us that eigenvectors corresponding to distinct
eigenvalues are orthogonal. Because \(A\) is (orthogonally) diagonalizable, we know the
geometric multiplicity of each eigenvalue is equal to its algebraic multiplicity. This
implies that we can use Gram-Schmidt on each eigenspace of dimension \(> 1\) to get a full
set of \(n\) orthogonal eigenvectors.
If we are willing to replace “diagonal” by “upper triangular” in the real spectral
theorem, we can weaken the requirement that \(A\) is symmetric to insisting only that \(A\)
has real eigenvalues.
Schur Triangularization Theorem If \(A\) is an \(n \times n\) matrix with \(n\) real eigenvalues, an
orthogonal matrix \(Q\) exists such that \(Q^{T}AQ\) is upper triangular.
There is also a lower triangular version of this theorem.
See Practice Problem ??
The eigenvalues of an upper triangular matrix are displayed along the main diagonal.
Because \(A\) and \(Q^{T}AQ\) have the same determinant and trace whenever \(Q\) is orthogonal (for they
are similar matrices), Theorem 23 gives:
If \(A\) is an \(n \times n\) matrix with real eigenvalues \(\lambda _{1}, \lambda _{2}, \dots , \lambda _{n}\) (possibly not all distinct), then \(\det A = \lambda _{1}\lambda _{2} \dots \lambda _{n}\) and \(\mbox {tr} A = \lambda _{1} + \lambda _{2} + \dots + \lambda _{n}\).
This
corollary remains true even if the eigenvalues are not real.
Practice Problems
Normalize the columns to make the following matrix orthogonal. \(A = \begin{bmatrix} 1 & -4 \\ 2 & 2 \end{bmatrix}\)