Solutions Using Matrix Exponentials

$\newenvironment {prompt}{}{} \newcommand {\Sec }[2]{\section {#1}} \newcommand {\trademark }[0]{{R\!\!\!\!\!\bigcirc }} \newcommand {\EXER }[0]{\section *{Exercises}} \newcommand {\CEXER }[0]{} \newcommand {\TEXER }[0]{} \newcommand {\R }[0]{\mbox {$\Bbb {R}$}} \newcommand {\C }[0]{\mbox {$\Bbb {C}$}} \newcommand {\Z }[0]{\mbox {$\Bbb {Z}$}} \newcommand {\N }[0]{\mbox {$\Bbb {N}$}} \newcommand {\D }[0]{\mbox {{\bf D}}} \newcommand {\setmin }[0]{\;\mbox {--}\;} \newcommand {\Matlab }[0]{{M\small {AT\-LAB}} } \newcommand {\Matlabp }[0]{{M\small {AT\-LAB}}} \newcommand {\computer }[0]{\Matlab Instructions} \newcommand {\half }[0]{\mbox {$\frac {1}{2}$}} \newcommand {\compose }[0]{\raisebox {.15ex}{\mbox {{\scriptsize $\circ $}}}} \newcommand {\AND }[0]{\quad \mbox {and}\quad } \newcommand {\vect }[2]{\left (\begin {array}{c} #1_1 \\ \vdots \\ #1_{#2}\end {array}\right )} \newcommand {\mattwo }[4]{\left (\begin {array}{rr} #1 & #2\\ #3 &#4\end {array}\right )} \newcommand {\mattwoc }[4]{\left (\begin {array}{cc} #1 & #2\\ #3 &#4\end {array}\right )} \newcommand {\vectwo }[2]{\left (\begin {array}{r} #1 \\ #2\end {array}\right )} \newcommand {\vectwoc }[2]{\left (\begin {array}{c} #1 \\ #2\end {array}\right )} \newcommand {\inv }[0]{^{-1}} \newcommand {\CC }[0]{{\cal C}} \newcommand {\CCone }[0]{\CC ^1} \newcommand {\Span }[0]{{\rm span}} \newcommand {\rank }[0]{{\rm rank}} \newcommand {\trace }[0]{{\rm tr}} \newcommand {\RE }[0]{{\rm Re}} \newcommand {\IM }[0]{{\rm Im}} \newcommand {\nulls }[0]{{\rm null\;space}} \newcommand {\dps }[0]{\displaystyle } \newcommand {\arraystart }[0]{\renewcommand {\arraystretch }{1.8}} \newcommand {\arrayfinish }[0]{\renewcommand {\arraystretch }{1.2}} \newcommand {\Start }[1]{\vspace {0.08in}\noindent {\bf Section~\ref {#1}}} \newcommand {\exer }[1]{\noindent {\bf \ref {#1}}} \newcommand {\ans }[0]{} \newcommand {\matthree }[9]{\left (\begin {array}{rrr} #1 & #2 & #3 \\ #4 & #5 & #6 \\ #7 & #8 & #9\end {array}\right )} \newcommand {\cvectwo }[2]{\left (\begin {array}{c} #1 \\ #2\end {array}\right )} \newcommand {\cmatthree }[9]{\left (\begin {array}{ccc} #1 & #2 & #3 \\ #4 & #5 & #6 \\ #7 & #8 & #9\end {array}\right )} \newcommand {\vecthree }[3]{\left (\begin {array}{r} #1 \\ #2 \\ #3\end {array}\right )} \newcommand {\cvecthree }[3]{\left (\begin {array}{c} #1 \\ #2 \\ #3\end {array}\right )} \newcommand {\cmattwo }[4]{\left (\begin {array}{cc} #1 & #2\\ #3 &#4\end {array}\right )} \newcommand {\thehelp }[0]{\thesection .\arabic {equation}} \newcommand {\matlabEquation }[0]{\let \oldtheequation \theequation \renewcommand {\theequation }{\oldtheequation *}\begin {equation}} \newcommand {\HyperFirstAtBeginDocument }[0]{\AtBeginDocument } \newcommand {\epstopdfsetup }[0]{\setkeys {ETE}} changed the theorem header of amsthm failed\MessageBreak }}} \newcommand {\GetTitleStringSetup }[0]{\setkeys {gettitlestring}} \newcommand {\Sectionformat }[2]{#1} \newcommand {\arraystretch }[0]{1.8} \newcommand {\arraystretch }[0]{1.2} \newcommand {\arraystretch }[0]{1.8} \newcommand {\arraystretch }[0]{1.2}$ $% This file was *autogenerated* from solutionsUsingMatrixExponentials.sagetex.sage with % sagetex.py version 2015/08/26 v3.0-92d9f7a %228c1f0c58e72d6b961f9bb9acd937fc% md5sum of corresponding .sage file (minus "goboom","current_tex_line",and pause/unpause lines)$

In Section ?? we showed that the solution of the single ordinary differential equation $\dot x(t) = \lambda x(t)$ with initial condition $x(0)=x_0$ is $x(t) = e^{t\lambda }x_0$ (see (??) in Chapter ??). In this section we show that we may write solutions of systems of equations in a similar form. In particular, we show that the solution to the linear system of ODEs

$\begin{equation} \label{eq:x=Mx} \frac{dX}{dt} = CX \end{equation}$ with initial condition $X(0) = X_0,$ where $C$ is an $n\times n$ matrix and $X_0\in \R ^n$ , is $\begin{equation} \label{matrixsoln} X(t) = e^{tC}X_0. \end{equation}$

In order to make sense of the solution (??) we need to understand matrix exponentials. More precisely, since $tC$ is an $n\times n$ matrix for each $t\in \R$ , we need to make sense of the expression $e^L$ where $L$ is an $n\times n$ matrix. For this we recall the form of the exponential function as a power series: $e^t = 1 + t + \frac {1}{2!} t^2 + \frac {1}{3!} t^3 + \frac {1}{4!} t^4 + \cdots .$ In more compact notation we have $e^t = \sum \limits _{k=0}^\infty \frac {1}{k!} t^k.$ By analogy, define the matrix exponential $e^L$ by

$\begin{eqnarray} e^{L} & = & I_n + L + \frac{1}{2!} L^2 + \frac{1}{3!} L^3 +\cdots \label{e:expL}\\ & = & \sum\limits_{k=0}^\infty\frac{1}{k!} L^k. \nonumber \end{eqnarray}$

In this formula $L^2 = LL$ is the matrix product of $L$ with itself, and the power $L^k$ is defined inductively by $L^k = LL^{k-1}$ for $k>1$ . Hence $e^L$ is an $n\times n$ matrix and is the infinite sum of $n\times n$ matrices.

Remark: The infinite series for matrix exponentials (??) does converge for all $n\times n$ matrices $L$ , and this fact is proved in Exercises ?? and ??.

Using (??), we can write the matrix exponential of $tC$ for each real number $t$ . Since $(tC)^k = t^k C^k$ we obtain

$\begin{equation} \label{eq:MatrixExp} \begin{array}{rcl} \dps e^{tC} & = & \dps I_n + tC + \frac{1}{2!} (tC)^2 + \frac{1}{3!} (tC)^3 +\cdots\\ \dps & = & \dps I_n + tC + \frac{t^2}{2!} C^2 + \frac{t^3}{3!} C^3 +\cdots. \end{array} \end{equation}$ Next we claim that $\begin{equation} \label {e:diffmatexp} \frac{d}{dt} e^{tC} = Ce^{tC}. \end{equation}$ We verify the claim by supposing that we can differentiate (??) term by term with respect to $t$ . Then $\begin{align*} \dps\frac{d}{dt} e^{tC} & = \frac{d}{dt}(I_n) + \frac{d}{dt}(tC) + \frac{d}{dt}\left(\frac{t^2}{2!} C^2\right) + \frac{d}{dt}\left(\frac{t^3}{3!} C^3\right) + \\ &\quad \frac{d}{dt}\left(\frac{t^4}{4!}C^4\right) + \cdots\\ & = 0 + C + t C^2 + \frac{t^2}{2!} C^3 + \frac{t^3}{3!} C^4 + \cdots\\ & = C\left(I_n + tC + \frac{t^2}{2!} C^2 + \frac{t^3}{3!} C^3 +\cdots\right)\\ & = Ce^{tC}. \end{align*}$

It follows that the function $X(t) = e^{tC}X_0$ is a solution of (??) for each $X_0\in \R ^n$ ; that is, $\frac {d}{dt} X(t) = \frac {d}{dt} e^{tC}X_0 = C e^{tC}X_0 = C X(t).$ Since (??) implies that $e^{0C} = e^0 = I_n$ , it follows that $X(t) = e^{tC}X_0$ is a solution of (??) with initial condition $X(0)=X_0$ . This discussion shows that solving (??) in closed form is equivalent to finding a closed form expression for the matrix exponential $e^{tC}$ .

The unique solution to the initial value problem $\begin {array}{rcl} \dps \frac {dX}{dt} & = & CX \\ X(0) & = & X_0 \end {array}$ is $X(t)=e^{tC}X_0.$

Proof: Existence follows from the previous discussion; uniqueness follows from the $n$ dimensional analog of Theorem ??. $\blacksquare$

Explicit Computation of Matrix Exponentials

We begin with the simplest computation of a matrix exponential.

(a) Let $L$ be a multiple of the identity; that is, let $L = \alpha I_n$ where $\alpha$ is a real number. Then

$\begin{equation} \label{ex:expm} e^{\alpha I_n} = e^{\alpha} I_n. \end{equation}$ That is, $e^{\alpha I_n}$ is a scalar multiple of the identity. To verify (??), compute $\begin{align*} e^{\alpha I_n} &= I_n + \alpha I_n + \frac{\alpha^2}{2!} I_n^2 + \frac{\alpha^3}{3!} I_n^3 +\cdots \\ &= (1+\alpha+\frac{\alpha^2}{2!} +\frac{\alpha^3}{3!}+\cdots)I_n = e^{\alpha} I_n. \end{align*}$

(b) Let $C$ be a $2\times 2$ diagonal matrix, $C = \mattwo {\lambda _1}{0}{0}{\lambda _2},$ where $\lambda _1$ and $\lambda _2$ are real constants. Then

$\begin{equation} \label{e:expdiag} e^{tC} = \mattwo{e^{\lambda_1 t}}{0}{0}{e^{\lambda_2 t}}. \end{equation}$ To verify (??) compute

$\begin{eqnarray*} e^{tC} & = & I_2 + tC + \frac{t^2}{2!} C^2 + \frac{t^3}{3!} C^3 +\cdots\\ & = & \mattwo{1}{0}{0}{1} + \mattwo{\lambda_1 t}{0}{0}{\lambda_2 t} + \mattwo{\frac{t^2}{2!}\lambda_1^2}{0}{0}{\frac{t^2}{2!}\lambda_2^2} +\cdots\\ & = & \mattwo{e^{\lambda_1 t}}{0}{0}{e^{\lambda_2 t}}. \end{eqnarray*}$

$\begin{equation} \label{e:exprotate} e^{tC} = \mattwo{\cos t}{-\sin t}{\sin t}{\cos t}. \end{equation}$ We begin this computation by observing that $C^2 = -I_2, \quad C^3 = -C, \AND C^4 = I_n.$ Therefore, by collecting terms of odd and even power in the series expansion for the matrix exponential we obtain $\begin{align*} e^{tC} & = I_2 + tC + \frac{t^2}{2!} C^2 + \frac{t^3}{3!}C^3 +\cdots\\ & = I_2 + tC - \frac{t^2}{2!}I_2 - \frac{t^3}{3!}C +\cdots\\ & = \left(1 - \frac{t^2}{2!} + \frac{t^4}{4!} - \frac{t^6}{6!} + \cdots \right)I_2 + \\ &\quad \left(t - \frac{t^3}{3!} + \frac{t^5}{5!} - \frac{t^7}{7!} + \cdots \right)C \\ & = (\cos t)I_2 + (\sin t)C \\ & = \mattwo{\cos t}{-\sin t}{\sin t}{\cos t}. \end{align*}$

In this computation we have used the fact that the trigonometric functions $\cos t$ and $\sin t$ have the power series expansions:

$\begin{eqnarray*} \cos t & = & 1-\frac{1}{2!}t^2+\frac{1}{4!} t^4 + \cdots = \sum\limits_{k=0}^\infty\frac{(-1)^k}{(2k)!} t^{2k},\\ \sin t & = & t-\frac{1}{3!} t^3 + \frac{1}{5!} t^5 + \cdots = \sum\limits_{k=0}^\infty \frac{(-1)^k}{(2k+1)!} t^{2k+1}. \end{eqnarray*}$

See Exercise ?? for an alternative proof of (??).

To compute the matrix exponential MATLAB provides the command expm. We use this command to compute the matrix exponential $e^{tC}$ for $C=\mattwo {0}{-1}{1}{0} \AND t=\frac {\pi }{4}.$ Type

C = [0, -1; 1, 0];
 
t = pi/4;
 
expm(t*C)

that gives the answer

ans =
0.7071 -0.7071
0.7071 0.7071

Indeed, this is precisely what we expect by (??), since $\cos \left (\frac {\pi }{4}\right )=\sin \left (\frac {\pi }{4}\right )= \frac {1}{\sqrt {2}}\approx 0.70710678.$

(d) Let $C = \mattwo {0}{1}{0}{0}.$ Then

$\begin{equation} \label{e:nilpotent} e^{tC} = I_2 + tC = \mattwo{1}{t}{0}{1}, \end{equation}$ since $C^2=0$ .

Exercises

Let $L$ be the $3\times 3$ matrix $L = \left (\begin {array}{rrr} 2 & 0 & -1\\ 0 & -1 & 3\\ 1 & 0 & 1 \end {array}\right ).$ Find the smallest integer $m$ such that $I_3+L+\frac {1}{2!} L^2 + \frac {1}{3!} L^3 + \cdots + \frac {1}{m!} L^m$ is equal to $e^L$ up to a precision of two decimal places. More exactly, use the MATLAB command expm to compute $e^L$ and use MATLAB commands to compute the series expansion to order $m$ . Note that the command for computing $n!$ in MATLAB is prod(1:n).

Use MATLAB to compute the matrix exponential $e^{tC}$ for $C =\mattwo {1}{1}{2}{-1}$ by choosing for $t$ the values $1.0,1.5$ and $2.5$ . Does $e^{C} e^{1.5C}=e^{2.5C}$ ?

For the scalar exponential function $e^{t}$ it is well known that for any pair of real numbers $t_1,t_2$ the following equality holds: $e^{t_1+t_2} = e^{t_1}e^{t_2}.$ Use MATLAB to find two $2\times 2$ matrices $C_1$ and $C_2$ such that $e^{C_1+C_2} \not = e^{C_1}e^{C_2}.$

In Exercises ?? – ?? compute the matrix exponential $e^{tC}$ for the matrix.

$\mattwo {0}{1}{0}{0}$ .

$\left (\begin {array}{ccc} 0 & 1 & 0\\ 0 & 0 & 1\\ 0 & 0 & 0 \end {array}\right )$ .

$\mattwo {0}{-2}{2}{0}$ .

Let $\alpha ,\beta$ be real numbers and let $\alpha I$ and $\beta I$ be corresponding $n\times n$ diagonal matrices. Use properties of the scalar exponential function to show that $e^{(\alpha + \beta )I} = e^{\alpha I}e^{\beta I}.$

In Exercises ?? – ?? we use Theorem ??, the uniqueness of solutions to initial value problems, in perhaps a surprising way.

Prove that $e^{t+s} = e^te^s$ for all real numbers $s$ and $t$ . Hint:

Fix $s$ and verify that $y(t) = e^{t+s}$ is a solution to the initial value problem $\begin{equation} \label{E:init1} \begin{array}{rcl} \frac{dx}{dt} & = & x \\ x(0) & = & e^s \end{array} \end{equation}$
Fix $s$ and verify that $z(t) = e^te^s$ is also a solution to (??).
Use Theorem ?? to conclude that $y(t)=z(t)$ for every $s$ .

Let $A$ be an $n\times n$ matrix. Prove that $e^{(t+s)A} = e^{tA}e^{sA}$ for all real numbers $s$ and $t$ . Hint:

Fix $s\in \R$ and $X_0\in \R ^n$ and verify that $Y(t) = e^{(t+s)A}X_0$ is a solution to the initial value problem $\begin{equation} \label{E:init2} \begin{array}{rcl} \frac{dX}{dt} & = & AX \\ X(0) & = & e^{sA}X_0 \end{array} \end{equation}$
Fix $s$ and verify that $Z(t) = e^{tA}\left (e^{sA}X_0\right )$ is also a solution to (??).
Use the $n$ dimensional version of Theorem ?? to conclude that $Y(t)=Z(t)$ for every $s$ and every $X_0$ .

Remark: Compare the result in this exercise with the calculation in Exercise ??.

Prove that $\begin{equation} \label{E:0-110E} \exp\left(t\mattwo{0}{-1}{1}{0}\right) = \mattwo{\cos t}{-\sin t}{\sin t}{\cos t}. \end{equation}$ Hint:

Verify that $X_1(t) = \vectwo {\cos t}{\sin t}$ and $X_2(t) = \vectwo {-\sin t}{\cos t}$ are solutions to the initial value problems $\begin{equation} \label{E:init3} \begin{array}{rcl} \dps\frac{dX}{dt} & = & \mattwo{0}{-1}{1}{0}X \\ X(0) & = & e_j \end{array} \end{equation}$ for $j=1,2$ .
Since $X_j(0)=e_j$ , use Theorems ?? and ?? to verify that $\begin{equation} \label{E:0-110} X_j(t) = \exp\left(t\mattwo{0}{-1}{1}{0}\right)e_j. \end{equation}$
Show that (??) proves (??)

Let $C$ be an $n\times n$ matrix. Use Theorem ?? to show that the $n$ columns of the $n\times n$ matrix $e^{tC}$ give a basis of solutions for the system of differential equations $\dot {X}=CX$ .

Remark: The completion of Exercises ?? and ?? constitutes a proof that the infinite series definition of the matrix exponential is a convergent series for all $n\times n$ matrices.

Let $A=(a_{ij})$ be an $n\times n$ matrix. Define $||A||_m = \max _{1\leq i\leq n} (|a_{i1}|+\cdots +|a_{in}|) = \max _{1\leq i\leq n} \left (\sum _{j=1}^n|a_{ij}|\right ).$ That is, to compute $||A||_m$ , first sum the absolute values of the entries in each row of $A$ , and then take the maximum of these sums. Prove that: $||AB||_m \leq ||A||_m ||B||_m.$ Hint: Begin by noting that $\begin{align*} ||AB||_m &= \max_{1\leq i\leq n}\left(\sum_{j=1}^n\left|\sum_{k=1}^na_{ik}b_{kj}\right| \right)\leq \max_{1\leq i\leq n}\left(\sum_{j=1}^n\sum_{k=1}^n\left|a_{ik}b_{kj} \right|\right) \\ &= \max_{1\leq i\leq n}\left(\sum_{k=1}^n\sum_{j=1}^n \left|a_{ik}b_{kj}\right|\right). \end{align*}$

Recall that an infinite series of real numbers $c_1+c_2 +\cdots +c_N + \cdots$ converges absolutely if there is a constant $K$ such that for every $N$ the partial sum satisfies: $|c_1| + |c_2| + \cdots + |c_N| \leq K.$

Let $A$ be an $n\times n$ matrix. To prove that the matrix exponential $e^A$ is an absolutely convergent infinite series use Exercise ?? and the following steps. Let $a_N$ be the $(i,j)^{th}$ entry in the matrix $A^N$ where $A^0=I_n$ .

$|a_N| \leq ||A^N||_m$ .
$||A^N||_m \leq ||A||_m^N$ .
$|a_0| + |a_1| + \cdots + \frac {1}{N!}|a_N| \leq e^{||A||_m}$ .