Symmetric Matrices

$\newenvironment {prompt}{}{} \newcommand {\Sec }[2]{\section {#1}} \newcommand {\trademark }[0]{{R\!\!\!\!\!\bigcirc }} \newcommand {\EXER }[0]{\section *{Exercises}} \newcommand {\CEXER }[0]{} \newcommand {\TEXER }[0]{} \newcommand {\R }[0]{\mbox {$\Bbb {R}$}} \newcommand {\C }[0]{\mbox {$\Bbb {C}$}} \newcommand {\Z }[0]{\mbox {$\Bbb {Z}$}} \newcommand {\N }[0]{\mbox {$\Bbb {N}$}} \newcommand {\D }[0]{\mbox {{\bf D}}} \newcommand {\setmin }[0]{\;\mbox {--}\;} \newcommand {\Matlab }[0]{{M\small {AT\-LAB}} } \newcommand {\Matlabp }[0]{{M\small {AT\-LAB}}} \newcommand {\computer }[0]{\Matlab Instructions} \newcommand {\half }[0]{\mbox {$\frac {1}{2}$}} \newcommand {\compose }[0]{\raisebox {.15ex}{\mbox {{\scriptsize $\circ $}}}} \newcommand {\AND }[0]{\quad \mbox {and}\quad } \newcommand {\vect }[2]{\left (\begin {array}{c} #1_1 \\ \vdots \\ #1_{#2}\end {array}\right )} \newcommand {\mattwo }[4]{\left (\begin {array}{rr} #1 & #2\\ #3 &#4\end {array}\right )} \newcommand {\mattwoc }[4]{\left (\begin {array}{cc} #1 & #2\\ #3 &#4\end {array}\right )} \newcommand {\vectwo }[2]{\left (\begin {array}{r} #1 \\ #2\end {array}\right )} \newcommand {\vectwoc }[2]{\left (\begin {array}{c} #1 \\ #2\end {array}\right )} \newcommand {\inv }[0]{^{-1}} \newcommand {\CC }[0]{{\cal C}} \newcommand {\CCone }[0]{\CC ^1} \newcommand {\Span }[0]{{\rm span}} \newcommand {\rank }[0]{{\rm rank}} \newcommand {\trace }[0]{{\rm tr}} \newcommand {\RE }[0]{{\rm Re}} \newcommand {\IM }[0]{{\rm Im}} \newcommand {\nulls }[0]{{\rm null\;space}} \newcommand {\dps }[0]{\displaystyle } \newcommand {\arraystart }[0]{\renewcommand {\arraystretch }{1.8}} \newcommand {\arrayfinish }[0]{\renewcommand {\arraystretch }{1.2}} \newcommand {\Start }[1]{\vspace {0.08in}\noindent {\bf Section~\ref {#1}}} \newcommand {\exer }[1]{\noindent {\bf \ref {#1}}} \newcommand {\ans }[0]{} \newcommand {\matthree }[9]{\left (\begin {array}{rrr} #1 & #2 & #3 \\ #4 & #5 & #6 \\ #7 & #8 & #9\end {array}\right )} \newcommand {\cvectwo }[2]{\left (\begin {array}{c} #1 \\ #2\end {array}\right )} \newcommand {\cmatthree }[9]{\left (\begin {array}{ccc} #1 & #2 & #3 \\ #4 & #5 & #6 \\ #7 & #8 & #9\end {array}\right )} \newcommand {\vecthree }[3]{\left (\begin {array}{r} #1 \\ #2 \\ #3\end {array}\right )} \newcommand {\cvecthree }[3]{\left (\begin {array}{c} #1 \\ #2 \\ #3\end {array}\right )} \newcommand {\cmattwo }[4]{\left (\begin {array}{cc} #1 & #2\\ #3 &#4\end {array}\right )} \newcommand {\thehelp }[0]{\thesection .\arabic {equation}} \newcommand {\matlabEquation }[0]{\let \oldtheequation \theequation \renewcommand {\theequation }{\oldtheequation *}\begin {equation}} \newcommand {\HyperFirstAtBeginDocument }[0]{\AtBeginDocument } \newcommand {\epstopdfsetup }[0]{\setkeys {ETE}} changed the theorem header of amsthm failed\MessageBreak }}} \newcommand {\GetTitleStringSetup }[0]{\setkeys {gettitlestring}} \newcommand {\Sectionformat }[2]{#1}$ $% This file was *autogenerated* from symmetricMatrices.sagetex.sage with % sagetex.py version 2015/08/26 v3.0-92d9f7a %8cfc62e0f1da921dac95b366b8aac6f0% md5sum of corresponding .sage file (minus "goboom","current_tex_line",and pause/unpause lines)$

Symmetric matrices have some remarkable properties that can be summarized by:

Let $A$ be an $n\times n$ symmetric matrix. Then

(a): every eigenvalue of $A$ is real, and
(b): there is an orthonormal basis of $\R ^n$ consisting of eigenvectors of $A$ .

As a consequence of Theorem ??, let ${\cal V}=\{v_1,\ldots ,v_n\}$ be an orthonormal basis for $\R ^n$ consisting of eigenvectors of $A$ . Indeed, suppose $Av_j = \lambda _jv_j$ where $\lambda _j\in \R$ . Note that $Av_j\cdot v_i = \left \{\begin {array}{rl} \lambda _j & \qquad i=j\\ 0 & \qquad i\neq j \end {array}\right .$ It follows from (??) that $[A]_{\cal V}= \left (\begin {array}{ccc} \lambda _1 & & 0 \\ & \ddots & \\ 0 & & \lambda _n \end {array}\right )$ is a diagonal matrix. So every symmetric matrix is similar to a diagonal matrix.

Hermitian Inner Products

The proof of Theorem ?? uses the Hermitian inner product — a generalization of dot product to complex vectors. Let $v,w\in \C ^n$ be two complex $n$ -vectors. Define $\langle v,w \rangle = v_1\overline {w}_1 + \cdots + v_n\overline {w}_n.$ Note that the coordinates $w_i$ of the second vector enter this formula with a complex conjugate. However, if $v$ and $w$ are real vectors, then $\langle v,w \rangle = v\cdot w.$ A more compact notation for the Hermitian inner product is given by matrix multiplication. Suppose that $v$ and $w$ are column $n$ -vectors. Then $\langle v,w \rangle = v^t\overline {w}.$

The properties of the Hermitian inner product are similar to those of dot product. We note three. Let $c\in \C$ be a complex scalar. Then

$\begin{eqnarray*} \langle v,v \rangle & = & ||v||^2\ge 0\\ \langle cv,w \rangle & = & c\langle v,w \rangle \\ \langle v,cw \rangle & = & \overline{c} \langle v,w \rangle \end{eqnarray*}$

Note the complex conjugation of the complex scalar $c$ in the previous formula.

Let $C$ be a complex $n\times n$ matrix. Then the main observation concerning Hermitian inner products that we shall use is: $\langle Cv,w \rangle = \langle v,\overline {C}^tw \rangle .$ This fact is verified by calculating $\langle Cv,w \rangle = (Cv)^t\overline {w} = (v^tC^t)\overline {w} = v^t(C^t\overline {w}) = v^t(\overline {\overline {C}^tw}) = \langle v,\overline {C}^tw \rangle .$ So if $A$ is a $n\times n$ real symmetric matrix, then

$\begin{equation} \label{e:symminv} \langle Av,w \rangle = \langle v,Aw \rangle, \end{equation}$ since $\overline {A}^t= A^t = A$ .

Theorem ??(a): Let $\lambda$ be an eigenvalue of $A$ and let $v$ be the associated eigenvector. Since $Av=\lambda v$ we can use (??) to compute $\lambda \langle v,v \rangle = \langle Av,v \rangle = \langle v,Av \rangle = \overline {\lambda } \langle v,v \rangle .$ Since $\langle v,v \rangle = ||v||^2 > 0$ , it follows that $\lambda =\overline {\lambda }$ and $\lambda$ is real. $\blacksquare$

Theorem ??(b)

Let $A$ be a real symmetric $n\times n$ matrix. We want to show that there is an orthonormal basis of $\R ^n$ consisting of eigenvectors of $A$ . The proof proceeds inductively on $n$ . The theorem is trivially valid for $n=1$ ; so we assume that it is valid for $n-1$ .

Theorem ?? of Chapter ?? implies that $A$ has an eigenvalue $\lambda _1$ and Theorem ??(a) states that this eigenvalue is real. Let $v_1$ be a unit length eigenvector corresponding to the eigenvalue $\lambda _1$ . Extend $v_1$ to an orthonormal basis $v_1,w_2,\ldots ,w_n$ of $\R ^n$ and let $P=(v_1|w_2|\cdots |w_n)$ be the matrix whose columns are the vectors in this orthonormal basis. Orthonormality and direct multiplication implies that

$\begin{equation} \label{e:orthosym} P^tP=I_n. \end{equation}$ Therefore $P$ is invertible; indeed $P\inv =P^t$ .

Next, let $B= P\inv AP.$ By direct computation $Be_1 = P\inv APe_1 = P\inv Av_1 = \lambda _1 P\inv v_1=\lambda _1e_1.$ It follows that that $B$ has the form $B = \mattwo {\lambda _1}{*}{0}{C}$ where $C$ is an $(n-1)\times (n-1)$ matrix. Since $P\inv =P^t$ , it follows that $B$ is a symmetric matrix; to verify this point compute $B^t = (P^t AP)^t = P^t A^t (P^t)^t = P^tAP = B.$ It follows that $B =\mattwo {\lambda _1}{0}{0}{C}$ where $C$ is a symmetric matrix. By induction we can choose an orthonormal basis $z_2,\ldots ,z_n$ in $\{0\}\times \R ^{n-1}$ consisting of eigenvectors of $C$ . It follows that $e_1,z_2,\ldots ,z_n$ is an orthonormal basis for $\R ^n$ consisting of eigenvectors of $B$ .

Finally, let $v_j=P\inv z_j$ for $j=2,\ldots ,n$ . Since $v_1=P\inv e_1$ , it follows that $v_1,v_2,\ldots ,v_n$ is a basis of $\R ^n$ consisting of eigenvectors of $A$ . We need only show that the $v_j$ form an orthonormal basis of $\R ^n$ . This is done using (??). For notational convenience let $z_1=e_1$ and compute

$\begin{align*} \langle v_i,v_j \rangle &=\langle P\inv z_i,P\inv z_j\rangle = \langle P^tz_i, P^tz_j \rangle \\ &= \langle z_i, PP^t z_j \rangle = \langle z_i,z_j \rangle, \end{align*}$

since $PP^t= I_n$ . Thus the vectors $v_j$ form an orthonormal basis since the vectors $z_j$ form an orthonormal basis. $\blacksquare$

Exercises

Let $A = \mattwo {a}{b}{b}{d}$ be the general real $2\times 2$ symmetric matrix.

Prove directly using the discriminant of the characteristic polynomial that $A$ has real eigenvalues.
Show that $A$ has equal eigenvalues only if $A$ is a scalar multiple of $I_2$ .

Let $A=\mattwo {1}{2}{2}{-2}.$ Find the eigenvalues and eigenvectors of $A$ and verify that the eigenvectors are orthogonal.

In Exercises ?? – ?? compute the eigenvalues and the eigenvectors of the $2\times 2$ matrix. Then load the matrix into the program map in MATLAB and iterate. That is, choose an initial vector $v_0$ and use map to compute $v_1=Av_0$ , $v_2=Av_1$ , …. How does the result of iteration compare with the eigenvectors and eigenvalues that you have found? Hint: You may find it convenient to use the feature Rescale in the MAP Options. Then the norm of the vectors is rescaled to $1$ after each use of the command Map and the vectors $v_j$ will not escape from the viewing screen.

$A=\mattwo {1}{3}{3}{1}$

$B=\mattwo {11}{9}{9}{11}$

$C=\mattwo {0.005}{-2.005}{-2.005}{0.005}$

Perform the same computational experiment as described in Exercises ?? – ?? using the matrix $A=\mattwo {0}{2}{2}{0}$ and the program map. How do your results differ from the results in those exercises and why?