Orthogonal Matrices and Symmetric Matrices

Recall that an \(n \times n\) matrix \(A\) is diagonalizable if and only if it has \(n\) linearly independent eigenvectors. Moreover, the matrix \(P\) with these eigenvectors as columns is a diagonalizing matrix for \(A\), that is

\begin{equation*} P^{-1}AP =D \end{equation*}

where \(D\) is a diagonal matrix with diagonal entries consisting of the eigenvalues of \(A\).

Orthogonal Matrices

A collection of non-zero, pairwise orthogonal vectors in \(\RR ^n\) is called an orthogonal set of vectors. An orthogonal set of vectors is called orthonormal if \(\norm {\vec {q}} = 1\) for each vector \(\vec {q}\) in the set. A set of orthogonal vectors \(\{\vec {v}_{1}, \vec {v}_{2}, \dots , \vec {v}_{k}\}\) can be “normalized”, i.e. converted into an orthonormal set \(\left \{ \frac {1}{\norm {\vec {v}_{1}}}\vec {v}_{1}, \frac {1}{\norm {\vec {v}_{2}}}\vec {v}_{2}, \dots , \frac {1}{\norm {\vec {v}_{k}}}\vec {v}_{k} \right \}\). In particular, if a matrix \(A\) has \(n\) orthogonal eigenvectors, they can (by normalizing) be taken to be orthonormal. The corresponding diagonalizing matrix (we will use \(Q\) instead of \(P\)) has orthonormal columns, and such matrices are very easy to invert.

First note that condition (a) is equivalent to \(Q^{T}Q = I\). Let \(\vec {q}_{1}, \vec {q}_{2}, \dots , \vec {q}_{n}\) denote the columns of \(Q\). Then \(\vec {q}_{i}^{T}\) is the \(i\)th row of \(Q^{T}\), so the \((i, j)\)-entry of \(Q^{T}Q\) is \(\vec {q}_{i} \dotp \vec {q}_{j}\). Thus \(Q^{T}Q = I\) means that \(\vec {q}_{i} \dotp \vec {q}_{j} = 0\) if \(i \neq j\) and \(\vec {q}_{i} \dotp \vec {q}_{j} = 1\) if \(i = j\). Hence condition (a) is equivalent to (c). The proof of the equivalence of (a) and (b) is similar.

Let \(A=\begin{bmatrix} 2&1&1\\-1&1&1\\0&-1&1 \end{bmatrix}\)
(a)
Check that matrix \(A\) has rows that are orthogonal.
(b)
Check that matrix \(A\) has columns that are NOT orthogonal.
(c)
Check that matrix \(A\) has rows that are NOT orthonormal.
(d)
Create a matrix \(Q\) by normalizing each of the rows of \(A\).
(e)
Check that columns and rows of \(Q\) are orthonormal.

You should get \(Q = \begin{bmatrix} \frac {2}{\sqrt {6}} & \frac {1}{\sqrt {6}} & \frac {1}{\sqrt {6}} \\ \frac {-1}{\sqrt {3}} & \frac {1}{\sqrt {3}} & \frac {1}{\sqrt {3}} \\ 0 & \frac {-1}{\sqrt {2}} & \frac {1}{\sqrt {2}} \end{bmatrix}\).

We have learned earlier that when we diagonalize a matrix \(A\), we write \(P^{-1}AP=D\) for some matrix \(P\) where \(D\) is diagonal, and the diagonal entries are the eigenvalues of \(A\). We have also learned that the columns of the matrix \(P\) are the corresponding eigenvectors of \(A\). So when a matrix is orthogonally diagonalizable, we are able to accomplish the diagonalization using a matrix \(Q\) consisting of \(n\) eigenvectors that form an orthonormal basis for \(\RR ^n\). In the following section we will learn that the matrices that have this property are precisely the symmetric matrices.

Symmetric Matrices

A symmetric matrix is a matrix which is equal to its transpose.

When we began our study of eigenvalues and eigenvectors, we saw examples of matrices with entries that were real numbers with eigenvalues that were complex numbers. It can be shown that symmetric matrices only have real eigenvalues.

Recall that \(\vec {x} \dotp \vec {y} = \vec {x}^{T} \vec {y}\) for all columns \(\vec {x}\) and \(\vec {y}\). Because \(A^{T} = A\), we get

\begin{equation*} (A\vec {x}) \dotp \vec {y} = (A\vec {x})^T\vec {y} = \vec {x}^TA^T\vec {y} = \vec {x}^TA\vec {y} = \vec {x} \dotp (A\vec {y}) \end{equation*}

Let \(A\vec {x} = \lambda \vec {x}\) and \(A\vec {y} = \mu \vec {y}\), where \(\lambda \neq \mu \). We compute

\begin{equation*} \lambda (\vec {x} \dotp \vec {y}) = (\lambda \vec {x}) \dotp \vec {y} = (A\vec {x}) \dotp \vec {y} = \vec {x} \dotp (A\vec {y}) = \vec {x} \dotp (\mu \vec {y}) = \mu (\vec {x} \dotp \vec {y}) \end{equation*}
Hence \((\lambda - \mu )(\vec {x} \dotp \vec {y}) = 0\), and so \(\vec {x} \dotp \vec {y} = 0\) because \(\lambda \neq \mu \).

Practice Problems

Normalize the columns to make the following matrix orthogonal. \(A = \begin{bmatrix} 1 & -4 \\ 2 & 2 \end{bmatrix}\)
\[\frac {1}{\sqrt {\answer {5}}}\begin{bmatrix}\answer {1} & \answer {-2}\\\answer {2} & \answer {1}\end{bmatrix}\]
Normalize the columns to make the following matrix orthogonal. \(A = \begin{bmatrix} -1 & 2 & 2 \\ 2 & -1 & 2 \\ 2 & 2 & -1 \end{bmatrix}\)
\[\frac {1}{\answer {3}}\begin{bmatrix}\answer {-1} & \answer {2} & \answer {2} \\ \answer {2} & \answer {-1} & \answer {2} \\ \answer {2} & \answer {2} & \answer {-1} \end{bmatrix}\]
For each matrix \(A\), find an orthogonal matrix \(Q\) such that \(Q^{-1}AQ\) is diagonal.
\[A = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\]
\[Q=\frac {1}{\sqrt {\answer {2}}}\begin{bmatrix}1 & -1\\\answer {1} & \answer {1}\end{bmatrix}\]
\[A = \begin{bmatrix} 3 & 0 & 7 \\ 0 & 5 & 0 \\ 7 & 0 & 3 \end{bmatrix}\]
List eigenvalues in increasing order.
\[\lambda _1=\answer {-4},\quad \lambda _2=\answer {5},\quad \lambda _3=\answer {10}\]
\[Q^{-1}AQ=\begin{bmatrix}\answer {-1}/\sqrt {\answer {2}} & \answer {0} & \answer {1}/\sqrt {\answer {2}}\\\answer {0} & \answer {1} & \answer {0}\\1/\sqrt {\answer {2}} & \answer {0} & 1/\sqrt {\answer {2}}\end{bmatrix}\begin{bmatrix}\lambda _1 & 0 & 0\\0 & \lambda _2 & 0\\0 & 0& \lambda _3\end{bmatrix}\begin{bmatrix}\answer {-1}/\sqrt {\answer {2}} & \answer {0} & \answer {1}/\sqrt {\answer {2}}\\\answer {0} & \answer {1} & \answer {0}\\1/\sqrt {\answer {2}} & \answer {0} & 1/\sqrt {\answer {2}}\end{bmatrix}\]
Suppose \(A\) is orthogonally diagonalizable. Prove that \(A\) is symmetric. (This is the easy direction of the "if and only if" in Theorem ??.)
Prove the converse of Theorem 9:

If \((A\vec {x}) \dotp \vec {y} = \vec {x} \dotp (A\vec {y})\) for all \(n\)-columns \(\vec {x}\) and \(\vec {y}\), then \(A\) is symmetric.

Let \(Q\) be an orthogonal matrix.
(a)
Show that \(\det Q = 1\) or \(\det Q = -1\).
(b)
Give \(2 \times 2\) examples of \(Q\) such that \(\det Q = 1\) and \(\det Q = -1\).

Text Source

This section was adapted from Section 8.2 of Keith Nicholson’s Linear Algebra with Applications. (CC-BY-NC-SA)

W. Keith Nicholson, Linear Algebra with Applications, Lyryx 2018, Open Edition, p. 424