The transpose of the matrix of cofactors of a matrix - it is part of a formula for the inverse of a matrix.

The multiplicity of an eigenvalue as a root of the characteristic equation.

Given any linear system $A\vec {x}=\vec {b}$, the system $A\vec {x}=\vec {0}$ is called the associated homogeneous system.

Every linear system $$\begin {array}{ccccccccc} a_{11}x_1 &+ &a_{12}x_2&+&\ldots &+&a_{1n}x_n&= &b_1 \\ a_{21}x_1 &+ &a_{22}x_2&+&\ldots &+&a_{2n}x_n&= &b_2 \\ &&&&\vdots &&&& \\ a_{m1}x_1 &+ &a_{m2}x_2&+&\ldots &+&a_{mn}x_n&= &b_m \end {array}$$ can be written in the augmented matrix form as follows: $$\left [\begin {array}{cccc|c} a_{11}&a_{12}&\ldots &a_{1n}&b_1\\a_{21}&a_{22}&\ldots &a_{2n}&b_2\\\vdots &\vdots &\ddots &\vdots &\vdots \\a_{m1}&a_{m2}&\ldots &a_{mn}&b_m \end {array}\right ]$$ The array to the left of the vertical bar is called the coefficient matrix of the linear system and is often given a capital letter name, like $A$. The vertical array to the right of the bar is called a constant vector. $$A=\begin {bmatrix}a_{11}&a_{12}&\ldots &a_{1n}\\a_{21}&a_{22}&\ldots &a_{2n}\\\vdots &\vdots &\ddots &\vdots \\a_{m1}&a_{m2}&\ldots &a_{mn}\end {bmatrix}\quad \text {and}\quad \vec {b}=\begin {bmatrix}b_1\\b_2\\\vdots \\b_m\end {bmatrix}$$ We will sometimes use the following notation to represent an augmented matrix.

$$\left [\begin {array}{c|c} A & \vec {b}\\ \end {array}\right ]$$

When a matrix is in row-echelon form, we can compute the solution to the system by starting from the last equation and working backwards. This process is known as back substitution.

When a coefficient matrix is in row echelon form, a basic variable is a variable corresponding to a column of the matrix with at least one leading entry.

A set $\mathcal {S}$ of vectors is called a basis of $\RR ^n$ (or a basis of a subspace $V$ of $\RR ^n$) provided that

(a)

$\mbox {span}(\mathcal {S})=\RR ^n$ (or $V$)

(b)

$\mathcal {S}$ is linearly independent.

Subdividing a matrix into submatrices using imaginary horizontal and vertical lines - used to multiply matrices more efficiently.

Matrix $A$ of Theorem th:matlintransgeneral is called the matrix of $T$ with respect to ordered bases $\mathcal {B}$ and $\mathcal {C}$.

The equation $$\mbox {det}(A-\lambda I) = 0$$ is called the characteristic equation of $A$.

The polynomial $$\mbox {det}(A-\lambda I)$$ is called the characteristic equation of $A$.

A set $V$ is said to be closed under addition if for each element $\vec {u} \in V$ and $\vec {v} \in V$ the sum $\vec {u}+\vec {v}$ is also in $V$.

A set $V$ is said to be closed under scalar multiplication if for each element $\vec {v} \in V$ and for each scalar $k \in \RR$ the product $k\vec {v}$ is also in $V$.

A coefficient matrix is a matrix whose entries are the coefficients of a system of linear equations. For the system $$\begin {array}{ccccccccc} a_{11}x_1 &+ &a_{12}x_2&+&\ldots &+&a_{1n}x_n&= &b_1 \\ a_{21}x_1 &+ &a_{22}x_2&+&\ldots &+&a_{2n}x_n&= &b_2 \\ &&&&\vdots &&&& \\ a_{m1}x_1 &+ &a_{m2}x_2&+&\ldots &+&a_{mn}x_n&= &b_m \end {array},$$ the coefficient matrix is $A=\begin {bmatrix}a_{11}&a_{12}&\ldots &a_{1n}\\a_{21}&a_{22}&\ldots &a_{2n}\\\vdots &\vdots &\ddots &\vdots \\a_{m1}&a_{m2}&\ldots &a_{mn}\end {bmatrix}$.

A method to compute $\mbox {det} A$ using determinants of minor matrices associated with one row or one column.

A matrix with $m$ rows and only 1 column.

Let $A$ be an $m\times n$ matrix. The column space of $A$, denoted by $\mbox {col}(A)$, is the subspace of $\RR ^m$ spanned by the columns of $A$.

Let $U$, $V$ and $W$ be vector spaces, and let $T:U\rightarrow V$ and $S:V\rightarrow W$ be linear transformations. The composition of $S$ and $T$ is the transformation $S\circ T:U\rightarrow W$ given by $$(S\circ T)(\vec {u})=S(T(\vec {u}))$$

The matrix of a composition is the product of the matrices corresponding to the transformations in the composition, in the same order.

A system of equations that has at least one solution.

when the iterates of an iterative method approach a solution

Let $\mathcal {B} = \{\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k\}$ be an ordered basis. Then the coordinate vector $\vec {v}$ is the column vector $\begin {bmatrix}c_1\\ c_2\\ \vdots \\c_k\end {bmatrix}$ such that $\vec {v} = c_1\vec {v}_1+c_2\vec {v}_2+\ldots +c_p\vec {v}_k$.

A method of solving systems of equations that uses determinants.

Let $\vec {u=\begin {bmatrix}u_1\\u_2\\u_3\end {bmatrix}}$ and $\vec {v}=\begin {bmatrix}v_1\\v_2\\v_3\end {bmatrix}$ be vectors in $\RR ^3$. The cross product of $\vec {u}$ and $\vec {v}$, denoted by $\vec {u}\times \vec {v}$, is given by $$\vec {u}\times \vec {v}=(u_2v_3-u_3v_2)\vec {i}-(u_1v_3-u_3v_1)\vec {j}+(u_1v_2-u_2v_1)\vec {k} =\begin {bmatrix}u_2v_3-u_3v_2\\-u_1v_3+u_3v_1\\u_1v_2-u_2v_1\end {bmatrix}$$

A function that assigns a scalar output to each square matrix $A$, denoted $\mbox {det} A$ - it is nonzero if and only if $A$ is invertible. Geometrically speaking, the determinant of a linear transformation of a square matrix is the factor by which area (or volume or hypervolume) is scaled by the transformation.

A matrix $A=[a_{ij}]$ where $a_{ij}=0$ whenever $i \ne j$

Let $A$ be an $n\times n$ matrix. Then $A$ is said to be diagonalizable if there exists an invertible matrix $P$ such that \begin{equation*} P^{-1}AP=D \end{equation*} where $D$ is a diagonal matrix. In other words, a matrix $A$ is diagonalizable if it is similar to a diagonal matrix, $A \sim D$.

Let $V$ be a subspace of $\RR ^n$. The dimension of $V$ is the number, $m$, of elements in any basis of $V$. We write $$\mbox {dim}(V)=m$$

An $m \times n$ matrix is a matrix with $m$ rows and $n$ columns.

Let $A(a_1, a_2,\ldots ,a_n)$ and $B(b_1, b_2,\ldots ,b_n)$ be points in $\RR ^n$. The distance between $A$ and $B$ is given by $$AB=\sqrt {(a_1-b_1)^2+(a_2-b_2)^2+\ldots +(a_n-b_n)^2}$$

when the iterates of an iterative method fail to approach a solution

An eigenvalue $\lambda$ of an $n \times n$ matrix $A$ is called a dominant eigenvalue if $\lambda$ has multiplicity $1$, and \begin{equation*} |\lambda | > |\mu | \quad \mbox{ for all eigenvalues } \mu \neq \lambda \end{equation*} Any corresponding eigenvector is called a dominant eigenvector of $A$.

Let $\vec {u}$ and $\vec {v}$ be vectors in $\RR ^n$. The dot product of $\vec {u}$ and $\vec {v}$, denoted by $\vec {u}\dotp \vec {v}$, is given by $$\vec {u}\dotp \vec {v}=\begin {bmatrix}u_1\\u_2\\\vdots \\u_n\end {bmatrix}\dotp \begin {bmatrix}v_1\\v_2\\\vdots \\v_n\end {bmatrix}=u_1v_1+u_2v_2+\ldots +u_nv_n$$

If $\lambda$ is an eigenvalue of an $n \times n$ matrix, the set of all eigenvectors associated to $\lambda$ along with the zero vector is the eigenspace associated to $\lambda$. The eigenspace is a subspace of $\RR ^n$.

Let $A$ be an $n \times n$ matrix. We say that a scalar $\lambda$ is an eigenvalue of $A$ if $$A\vec {x} = \lambda \vec {x}$$ for some nonzero vector $x$. We say that $x$ is an eigenvector of $A$ associated with the eigenvalue $\lambda$.

If we are able to diagonalize $A$, say $A=PDP^{-1}$, we say that $PDP^{-1}$ is an eigenvalue decomposition of $A$.

Let $A$ be an $n \times n$ matrix. We say that a non-zero vector $\vec {x}$ is an eigenvector of $A$ if $$A\vec {x} = \lambda \vec {x}$$ for some scalar $\lambda$. We say that $\lambda$ is an eigenvalue of $A$ associated with the eigenvector $\vec {x}$.

An elementary matrix is a square matrix formed by applying a single elementary row operation to the identity matrix.

The following three operations performed on a linear system are called elementary row operations.

(a)

Switching the order of equations (rows) $i$ and $j$: $$R_i\leftrightarrow R_j$$

(b)

Multiplying both sides of equation (row) $i$ by the same non-zero constant, $k$, and replacing equation $i$ with the result: $$kR_i\rightarrow R_i$$

(c)

Adding $k$ times equation (row) $i$ to equation (row) $j$, and replacing equation $j$ with the result: $$R_j+kR_i\rightarrow R_j$$

Linear systems are called equivalent if they have the same solution set.

When a linear system is in row-echelon form, the variables corresponding to columns that do not have any leading coefficients (if there are any) are known as free variables.

$\mbox {null}(A)$ is the orthogonal complement of $\mbox {row}(A)$, and $\mbox {null}(A^T)$ is the orthogonal complement of $\mbox {col}(A)$

The process of using the elementary row operations on a matrix to transform it into reduced row-echelon form is called Gauss-Jordan elimination.

An iterative method for solving linear systems that is a refinement of the Jacobi method, where we use computed values of variables alternately for quicker convergence.

The process of using the elementary row operations on a matrix to transform it into row-echelon form is called Gaussian Elimination.

The geometric multiplicity of an eigenvalue $\lambda$ is the dimension of the corresponding eigenspace $\mathcal {S}_\lambda$.

A circle in the complex plane which has a diagonal entry of a matrix as its center and the sum of the absolute values of the other entries in that row (or column) as its radius.

Gershgorin’s theorem says that the $n$ eigenvalues of an $n \times n$ matrix can be found in the region in the complex plane consisting of the $n$ Gershgorin disks.

An iterative process which constructs an orthogonal basis for a subspace. The idea is to build the orthogonal set one vector at a time, by taking a vector not in the span of the vectors in the current iteration of the set, and subtracting its orthogonal projection onto each of those vectors.

A system of linear equations is called homogeneous if the system can be written in the form $$\begin {array}{ccccccccc} a_{11}x_1 &+ &a_{12}x_2&+&\ldots &+&a_{1n}x_n&= &0 \\ a_{21}x_1 &+ &a_{22}x_2&+&\ldots &+&a_{2n}x_n&= &0 \\ &&&&\vdots &&&& \\ a_{m1}x_1 &+ &a_{m2}x_2&+&\ldots &+&a_{mn}x_n&= &0 \end {array},$$ or as a matrix equation as $A \vec {x} = \vec {0}$.

A square matrix with ones as diagonal entries and zeros for the remaining entries.

The identity transformation on $V$, denoted by $\id _V$, is a transformation that maps each element of $V$ to itself.

In other words, $$\id _V:V\rightarrow V$$ is a transformation such that $$\id _V(\vec {v})=\vec {v}\quad \text {for all}\quad \vec {v} \in V$$

Let $V$ and $W$ be vector spaces, and let $T:V\rightarrow W$ be a linear transformation. The image of $T$, denoted by $\mbox {im}(T)$, is the set $$\mbox {im}(T)=\{T(\vec {v}):\vec {v}\in V\}$$ In other words, the image of $T$ consists of individual images of all vectors of $V$.

A system of equations that has no solution.

An inner product on a real vector space $V$ is a function that assigns a real number $\langle \vec {v}, \vec {w}\rangle$ to every pair $\vec {v}$, $\vec {w}$ of vectors in $V$ in such a way that the following properties are satisfied.

(a)

$\langle \vec {v}, \vec {w}\rangle$ is a real number for all $\vec {v}$ and $\vec {w}$ in $V$.

(b)

$\langle \vec {v}, \vec {w}\rangle = \langle \vec {w}, \vec {v}\rangle$ for all $\vec {v}$ and $\vec {w}$ in $V$.

(c)

$\langle \vec {v} + \vec {w}, \vec {u}\rangle = \langle \vec {v}, \vec {u}\rangle + \langle \vec {w}, \vec {u}\rangle$ for all $\vec {u}$, $\vec {v}$, and $\vec {w}$ in $V$.

(d)

$\langle r\vec {v}, \vec {w}\rangle = r\langle \vec {v}, \vec {w}\rangle$ for all $\vec {v}$ and $\vec {w}$ in $V$ and all $r$ in $\RR$.

(e)

$\langle \vec {v}, \vec {v}\rangle > 0$ for all $\vec {v} \neq \vec {0}$ in $V$.

Let $V$ and $W$ be vector spaces, and let $T:V\rightarrow W$ be a linear transformation. A transformation $S:W\rightarrow V$ that satisfies $S\circ T=\id _V$ and $T\circ S=\id _W$ is called an inverse of $T$. If $T$ has an inverse, $T$ is called invertible.

Let $A$ be an $n\times n$ matrix. An $n\times n$ matrix $B$ is called an inverse of $A$ if $$AB=BA=I$$ where $I$ is an $n\times n$ identity matrix. If such an inverse matrix exists, we say that $A$ is invertible. If an inverse does not exist, we say that $A$ is not invertible. The inverse of $A$ is denoted by $A^{-1}$.

Let $V$ and $W$ be vector spaces. If there exists an invertible linear transformation $T:V\rightarrow W$ we say that $V$ and $W$ are isomorphic and write $V\cong W$. The invertible linear transformation $T$ is called an isomorphism.

A technique where we repeat the same procedure (called an iteration) many times (usually using a computer), and we obtain approximate solutions which we hope “converge to” the actual solution.

An iterative method for solving a system of equations where one variable is isolated in each equation in order to compute the coordinate of the next iterate.

Let $V$ and $W$ be vector spaces, and let $T:V\rightarrow W$ be a linear transformation. The kernel of $T$, denoted by $\mbox {ker}(T)$, is the set $$\mbox {ker}(T)=\{\vec {v}:T(\vec {v})=\vec {0}\}$$ In other words, the kernel of $T$ consists of all vectors of $V$ that map to $\vec {0}$ in $W$.

The determinant of a matrix can be computed using cofactor expansion along ANY row or ANY column.

The first non-zero entry in a row of a matrix (when read from left to right) is called the leading entry. When the leading entry is 1, we refer to it as a leading 1.

When a coefficient matrix is in row echelon form, a leading variable is a variable corresponding to a column of the matrix with at least one leading entry.

A vector $\vec {v}$ is said to be a linear combination of vectors $\vec {v}_1, \vec {v}_2,\ldots , \vec {v}_n$ if $$\vec {v}=a_1\vec {v}_1+ a_2\vec {v}_2+\ldots + a_n\vec {v}_n$$ for some scalars $a_1, a_2, \ldots ,a_n$.

A linear equation in variables $x_1, \ldots , x_n$ is an equation that can be written in the form $$a_1x_1+a_2x_2+\ldots +a_nx_n=b$$ where $a_1,\ldots ,a_n$ and $b$ are constants.

A transformation $T:\RR ^n\rightarrow \RR ^m$ is called a linear transformation if the following are true for all vectors $\vec {u}$ and $\vec {v}$ in $\RR ^n$, and scalars $k$. \begin{equation} T(k\vec{u})= kT(\vec{u}) \end{equation} \begin{equation} T(\vec{u}+\vec{v})= T(\vec{u})+T(\vec{v}) \end{equation}

Let $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k$ be vectors of $\RR ^n$. We say that the set $\{\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k\}$ is linearly independent if the only solution to \begin{equation} c_1\vec{v}_1+c_2\vec{v}_2+\ldots +c_p\vec{v}_k=\vec{0} \end{equation} is the trivial solution $c_1=c_2=\ldots =c_k=0$.

If, in addition to the trivial solution, a non-trivial solution (not all $c_1, c_2,\ldots ,c_k$ are zero) exists, then we say that the set $\{\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k\}$ is linearly dependent.

Let $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k$ be vectors of $\RR ^n$. We say that the set $\{\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k\}$ is linearly independent if the only solution to \begin{equation} c_1\vec{v}_1+c_2\vec{v}_2+\ldots +c_p\vec{v}_k=\vec{0} \end{equation} is the trivial solution $c_1=c_2=\ldots =c_k=0$.

If, in addition to the trivial solution, a non-trivial solution (not all $c_1, c_2,\ldots ,c_k$ are zero) exists, then we say that the set $\{\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k\}$ is linearly dependent.

A factorization $A=LU$ where $L$ is lower triangular and $U$ is upper triangular with ones on the diagonal (called unit upper triangluar). It is useful for solving $Ax=b$.

Let $\vec {v}=\begin {bmatrix}v_1\\ v_2\\ \vdots \\v_n\end {bmatrix}$ be a vector in $\RR ^n$, then the length, or the magnitude, of $\vec {v}$ is given by $$\norm {\vec {v}}=\sqrt {v_1^2+v_2^2+\ldots +v_n^2}$$

A rectangular array of numbers. It has $m$ rows and $n$ columns for some positive integers $m$ and $n$.

Let $A=\begin {bmatrix} a_{ij}\end {bmatrix}$ and $B=\begin {bmatrix} b_{ij}\end {bmatrix}$ be two $m\times n$ matrices. Then the sum of matrices $A$ and $B$, denoted by $A+B$, is an $m \times n$ matrix given by $$A+B=\begin {bmatrix}a_{ij}+b_{ij}\end {bmatrix}$$

Let $A=\begin {bmatrix} a_{ij}\end {bmatrix}$ and $B=\begin {bmatrix} b_{ij}\end {bmatrix}$ be two $m \times n$ matrices. Then $A=B$ means that $a_{ij}=b_{ij}$ for all $1\leq i\leq m$ and $1\leq j\leq n$.

Representing a matrix as a product of two or more matrices.

Let $A$ be an $m\times n$ matrix whose rows are vectors $\vec {r}_1$, $\vec {r}_2,\ldots ,\vec {r}_n$. Let $B$ be an $n\times p$ matrix with columns $\vec {b}_1, \vec {b}_2, \ldots , \vec {b}_p$. Then the matrix product $AB$ is an $m \times p$ matrix with entries given by the dot products $$AB=\begin {bmatrix}-&\vec {r}_1&-\\-&\vec {r}_2&-\\ &\vdots & \\-&\vec {r}_i &-\\ &\vdots & \\-&\vec {r}_m&-\end {bmatrix}\begin {bmatrix}|&|&&|&&|\\\vec {b}_1& \vec {b}_2 &\ldots & \vec {b}_j&\ldots & \vec {b}_p\\|&|&&|&&|\end {bmatrix}=$$ $$=\begin {bmatrix}\vec {r}_1\dotp \vec {b}_1&\vec {r}_1\dotp \vec {b}_2&\ldots &\vec {r}_1\dotp \vec {b}_j&\ldots &\vec {r}_1\dotp \vec {b}_p\\\vec {r}_2\dotp \vec {b}_1&\vec {r}_2\dotp \vec {b}_2&\ldots &\vec {r}_2\dotp \vec {b}_j&\ldots &\vec {r}_2\dotp \vec {b}_p\\\vdots &\vdots &&\vdots &&\vdots \\\vec {r}_i\dotp \vec {b}_1&\vec {r}_i\dotp \vec {b}_2&\ldots &\vec {r}_i\dotp \vec {b}_j&\ldots &\vec {r}_i\dotp \vec {b}_p\\\vdots &\vdots &&\vdots &&\vdots \\\vec {r}_m\dotp \vec {b}_1&\vec {r}_m\dotp \vec {b}_2&\ldots &\vec {r}_m\dotp \vec {b}_j&\ldots &\vec {r}_m\dotp \vec {b}_p\end {bmatrix}$$

If $A=\begin {bmatrix} a_{ij}\end {bmatrix}$ and $k$ is a scalar, then $kA=\begin {bmatrix} ka_{ij}\end {bmatrix}$.

Let $A$ be an $m\times n$ matrix, and let $\vec {x}$ be an $n\times 1$ vector. The product $A\vec {x}$ is the $m\times 1$ vector given by: $$A\vec {x}=\begin {bmatrix} a_{11} & a_{12}&\dots &a_{1n}\\ a_{21}&a_{22} &\dots &a_{2n}\\ \vdots & \vdots &\ddots &\vdots \\ a_{m1}&\dots &\dots &a_{mn} \end {bmatrix}\begin {bmatrix}x_1\\x_2\\\vdots \\x_n\end {bmatrix}= x_1\begin {bmatrix}a_{11}\\a_{21}\\ \vdots \\a_{m1}\end {bmatrix}+ x_2\begin {bmatrix}a_{12}\\a_{22}\\ \vdots \\a_{m2}\end {bmatrix}+\dots + x_n\begin {bmatrix}a_{1n}\\a_{2n}\\ \vdots \\a_{mn}\end {bmatrix}$$ or, equivalently, $$A\vec {x}=\begin {bmatrix}a_{11}x_1+a_{12}x_2+\ldots +a_{1n}x_n\\a_{21}x_1+a_{22}x_2+\ldots +a_{2n}x_n\\\vdots \\a_{m1}x_1+a_{m2}x_2+\ldots +a_{mn}x_n\end {bmatrix}$$

Matrix $A$ of Theorem th:matlintransgeneral is called the matrix of $T$ with respect to ordered bases $\mathcal {B}$ and $\mathcal {C}$.

If $A$ is a square matrix then we can define $A^k$ to be the result of multiplying $A$ by itself $k$ times.

The additive inverse of a matrix, formed by multiplying the matrix by the scalar $-1$.

A square matrix $A$ is said to be nonsingular provided that $\mbox {rref}(A)=I$. Otherwise we say that $A$ is singular.

Let $A$ be an $m\times n$ matrix. The null space of $A$, denoted by $\mbox {null}(A)$, is the set of all vectors $\vec {x}$ in $\RR ^n$ such that $A\vec {x}=\vec {0}$. It is a subspace of $\RR ^n$.

The nullity of a linear transformation $T:V\rightarrow W$, is the dimension of the kernel of $T$. $$\mbox {nullity}(T)=\mbox {dim}(\mbox {ker}(T))$$

Let $A$ be a matrix. The dimension of the null space of $A$ is called the nullity of $A$. $$\mbox {dim}\Big (\mbox {null}(A)\Big )=\mbox {nullity}(A)$$

A linear transformation $T:V\rightarrow W$ is one-to-one if $$T(\vec {v}_1)=T(\vec {v}_2)\quad \text {implies that}\quad \vec {v}_1=\vec {v}_2$$

A linear transformation $T:V\rightarrow W$ is onto if for every element $\vec {w}$ of $W$, there exists an element $\vec {v}$ of $V$ such that $T(\vec {v})=\vec {w}$.

A basis in which the elements appear in a specific fixed order. Establishing an order is necessary because a coordinate vector with respect to a given basis relies on the order in which the basis elements appear.

A set of orthogonal vectors that spans a subspace. (Any orthogonal set of vectors must be linearly independent by Theorem orthbasis.)

If $W$ is a subspace, we define the orthogonal complement $W^\perp$ as the set of all vectors orthogonal to every vector in $W$, i.e., \begin{equation*} W^\perp = \{\vec{x} \in \RR ^n \mid \vec{x} \dotp \vec{y} = 0 \mbox{ for all } \vec{y} \in W\} \end{equation*}

Let $W$ be a subspace of $\RR ^n$ and let $\vec {x} \in \RR ^n$. Then there exist unique vectors $\vec {w} \in W$ and $\vec {w}^\perp \in W^\perp$ such that $\vec {x} = \vec {w} + \vec {w}^\perp$.

An $n \times n$ matrix $Q$ is called an orthogonal matrix if its columns form an orthonormal set. This will happen if and only if its rows form an orthonormal set. Note also that $Q$ is an orthogonal matrix if and only if it is an invertible matrix such that $Q^{-1}=Q^{T}$.

Let $W$ be a subspace of $\RR ^n$ with orthogonal basis $\{\vec {f}_{1}, \vec {f}_{2}, \dots , \vec {f}_{m}\}$. If $\vec {x}$ is in $\RR ^n$, the vector \begin{equation} \vec{w}=\mbox{proj}_W(\vec{x}) = \mbox{proj}_{\vec{f}_1}\vec{x} + \mbox{proj}_{\vec{f}_2}\vec{x} + \dots + \mbox{proj}_{\vec{f}_m}\vec{x} \end{equation} is called the orthogonal projection of $\vec {x}$ onto $W$.

Let $\vec {v}$ be a vector, and let $\vec {d}$ be a non-zero vector. The projection of $\vec {v}$ onto $\vec {d}$ is given by $$\mbox {proj}_{\vec {d}}\vec {v}=\left (\frac {\vec {v}\dotp \vec {d}}{\norm {\vec {d}}^2}\right )\vec {d}$$

Let $\{ \vec {v}_1, \vec {v}_2, \cdots , \vec {v}_k \}$ be a set of nonzero vectors in $\RR ^n$. Then this set is called an orthogonal set if $\vec {v}_i \dotp \vec {v}_j = 0$ for all $i \neq j$. Moreover, if $\norm {\vec {v}_i}=1$ for $i=1,\ldots ,m$ (i.e. each vector in the set is a unit vector), we say the set of vectors is an orthonormal set.

Let $\vec {u}$ and $\vec {v}$ be vectors in $\RR ^n$. We say $\vec {u}$ and $\vec {v}$ are orthogonal if $\vec {u}\dotp \vec {v}=0$.

An $n \times n$ matrix $A$ is said to be orthogonally diagonalizable if an orthogonal matrix $Q$ can be found such that $Q^{-1}AQ = Q^{T}AQ$ is diagonal.

A set of orthonormal vectors that spans a subspace. (Any orthogonal set of vectors must be linearly independent by Theorem orthbasis.)

Let $\{ \vec {v}_1, \vec {v}_2, \cdots , \vec {v}_k \}$ be a set of nonzero vectors in $\RR ^n$. Then this set is called an orthogonal set if $\vec {v}_i \dotp \vec {v}_j = 0$ for all $i \neq j$. Moreover, if $\norm {\vec {v}_i}=1$ for $i=1,\ldots ,m$ (i.e. each vector in the set is a unit vector), we say the set of vectors is an orthonormal set.

Let $\vec {v}=\begin {bmatrix}v_1\\v_2\\\vdots \\v_n\end {bmatrix}$ be a direction vector for line $l$ in $\RR ^n$, and let $(a_1, a_2,\ldots , a_n)$ be an arbitrary point on $l$. Then the following parametric equations describe $l$: $$x_1=v_1t+a_1$$ $$x_2=v_2t+a_2$$ $$\vdots$$ $$x_n=v_nt+a_n$$

Subdividing a matrix into submatrices using imaginary horizontal and vertical lines - used to multiply matrices more efficiently.

A matrix formed by permuting the rows of the identity matrix.

In Gaussian elimination, an entry chosen to become a leading coefficient used to get zeros in the remaining rows.

A square matrix is called positive definite if it is symmetric and all its eigenvalues $\lambda$ are positive. We write $\lambda >0$ when eigenvalues are real and positive.

The power method is an iterative method for computing the dominant eigenvalue of a matrix. It variants can compute the smallest eigenvalue or the eigenvalue closest to some target.

(a)

The determinant of a triangular matrix is the product of its diagonal entries.

(b)

The determinant of a matrix is equal to the determinant of its transpose.

(c)

The determinant of the inverse of a matrix is the reciprocal of the determinant of the matrix.

(d)

A matrix with a zero row has determinant zero.

(e)

Interchanging two rows of a matrix changes the sign of its determinant.

(f)

A matrix with two identical rows has determinant zero.

(g)

Multiplying a row of a matrix by $k$ changes the determinant by a factor of $k$.

(h)

Multiplying a matrix by $k$ changes the determinant by a factor of $k^n$.

(i)

Adding a multiple of one row of a matrix to another row does not change the determinant.

(j)

A matrix is singular if and only if its determinant is zero.

(k)

The determinant of a product is equal to the product of the determinants.

If $Q$ is an orthogonal matrix, then...

(a)

$Q^{-1} = Q^T$ is orthogonal,

(b)

$\mbox {det} Q = \pm 1$,

(c)

if $\lambda$ is an eigenvalue of $Q$, then $|\lambda |=1$,

(d)

the product of $Q$ with any other orthogonal matrix will be an orthogonal matrix (i.e. orthogonal matrices are closed under matrix multiplication),

and

(e)

$Q$ is a length-preserving and angle-preserving linear transformation.

Similar matrices must have the same...

(a)

determinant,

(b)

rank,

(c)

trace,

(d)

characteristic polynomial,

and

(e)

eigenvalues.

Let $A$ be an $m \times n$ matrix with independent columns. A QR-factorization of $A$ expresses it as $A = QR$ where $Q$ is $m \times n$ with orthonormal columns and $R$ is an invertible and upper triangular matrix with positive diagonal entries.

The rank of a linear transformation $T:V\rightarrow W$, is the dimension of the image of $T$. $$\mbox {rank}(T)=\mbox {dim}(\mbox {im}(T))$$

The rank of matrix $A$, denoted by $\mbox {rank}(A)$, is the number of nonzero rows that remain after we reduce $A$ to row-echelon form by elementary row operations.

For any matrix $A$, $$\mbox {rank}(A)=\mbox {dim}\Big (\mbox {row}(A)\Big )$$

Let $T:V\rightarrow W$ be a linear transformation. Suppose $\mbox {dim}(V)=n$, then $$\mbox {rank}(T)+\mbox {nullity}(T)=n$$

Let $A$ be an $m\times n$ matrix. Then $$\mbox {rank}(A)+\mbox {nullity}(A)=n$$

A matrix that is already in row-echelon form is said to be in reduced row-echelon form if:

(a)

Each leading entry is $1$

(b)

All entries above and below each leading $1$ are $0$

Let $\{\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\}$ be a set of vectors in $\RR ^n$. If we can remove one vector without changing the span of this set, then that vector is redundant. In other words, if $$\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\right )=\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_{j-1},\vec {v}_{j+1},\dots ,\vec {v}_k\right )$$ we say that $\vec {v}_j$ is a redundant element of $\{\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\}$, or simply redundant.

A matrix is said to be in row-echelon form if:

(a)

All entries below each leading entry are 0.

(b)

Each leading entry is in a column to the right of the leading entries in the rows above it.

(c)

All rows of zeros, if there are any, are located below non-zero rows.

Two matrices $A$ and $B$ are said to be row equivalent if there is a sequence of elementary row operations that converts $A$ to $B$.

A matrix with only $1$ row and $n$ columns.

Let $A$ be an $m\times n$ matrix. The row space of $A$, denoted by $\mbox {row}(A)$, is the subspace of $\RR ^n$ spanned by the rows of $A$.

A matrix with only $1$ row and $n$ columns.

If $A=\begin {bmatrix} a_{ij}\end {bmatrix}$ and $k$ is a scalar, then $kA=\begin {bmatrix} ka_{ij}\end {bmatrix}$.

If $A$ and $B$ are $n \times n$ matrices, we say that $A$ and $B$ are similar, if $B = P^{-1}AP$ for some invertible matrix $P$. In this case we write $A \sim B$.

A square matrix $A$ is said to be singular provided that $\mbox {rref}(A)$ is NOT the identity matrix. If instead $\mbox {rref}(A)=I$, we say that $A$ is nonsingular.

Let $A$ be an $m\times n$ matrix. The singular values of $A$ are the square roots of the positive eigenvalues of $A^TA.$

An $n\times n$ matrix $A$ is said to be symmetric if $A=A^{T}.$ It is said to be skew symmetric if $A=-A^{T}.$

Let $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$ be vectors in $\RR ^n$. The set $S$ of all linear combinations of $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$ is called the span of $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$. We write $$S=\mbox {span}(\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p)$$ and we say that vectors $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$ span $S$. Any vector in $S$ is said to be in the span of $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$. The set $\{\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p\}$ is called a spanning set for $S$.

Let $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$ be vectors in $\RR ^n$. The set $S$ of all linear combinations of $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$ is called the span of $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$. We write $$S=\mbox {span}(\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p)$$ and we say that vectors $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$ span $S$. Any vector in $S$ is said to be in the span of $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$. The set $\{\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p\}$ is called a spanning set for $S$.

If we are able to diagonalize $A$, say $A=PDP^{-1}$, we say that $PDP^{-1}$ is an eigenvalue decomposition of $A$.

If $A$ is a real $n \times n$ matrix, then $A$ is symmetric if an only if $A$ is orthogonally diagonalizable.

A matrix with the same number of rows and columns.

The set $\{\vec {e}_1, \ldots ,\vec {e}_n\}$ is called the standard basis of $\RR ^n$.

Let $T:\RR ^n\rightarrow \RR ^m$ be a linear transformation. Then the matrix \begin{equation*} \label{matlintrans} A=\begin{bmatrix} | & |& &|\\ T(\vec{e}_1) & T(\vec{e}_2)&\dots &T(\vec{e}_n)\\ |&| & &| \end{bmatrix} \end{equation*} is known as the standard matrix of the linear transformation $T$.

Let $\vec {e}_i$ denote a vector that has $1$ as the $i^{th}$ component and zeros elsewhere. In other words, $$\vec {e}_i=\begin {bmatrix} 0\\ 0\\ \vdots \\ 1\\ \vdots \\ 0 \end {bmatrix}$$ where $1$ is in the $i^{th}$ position. We say that $\vec {e}_i$ is a standard unit vector of $\RR ^n$.

Let $A=[a_{ij}]$ be the $n\times n$ matrix which is the coefficient matrix of the linear system $A \vec {x}= \vec {b}$. Let $$r_i(A):= \sum _{j \ne i} |a_{ij}|$$ denote the sum of the absolute values of the non-diagonal entries in row $i$. We say that $A$ is strictly diagonally dominant if $$|a_{ii}|>r_i(A)$$ for all values of $i$ from $i=1$ to $i=n$.

A nonempty subset $U$ of a vector space $V$ is called a subspace of $V$, provided that $U$ is itself a vector space when given the same addition and scalar multiplication as $V$.

Suppose that $V$ is a nonempty subset of $\RR ^n$ that is closed under addition and closed under scalar multiplication. Then $V$ is a subspace of $\RR ^n$.

Let $U$ be a nonempty subset of a vector space $V$. If $U$ is closed under the operations of addition and scalar multiplication of $V$, then $U$ is a subspace of $V$.

An $n\times n$ matrix $A$ is said to be symmetric if $A=A^{T}.$ It is said to be skew symmetric if $A=-A^{T}.$

A finite set of linear (degree 1) equations each with the same variables.

The trace of an $n \times n$ matrix $A$, abbreviated $\mbox {tr} A$, is defined to be the sum of the main diagonal elements of $A$. In other words, if $A = [a_{ij}]$, then $$\mbox {tr}(A) = a_{11} + a_{22} + \dots + a_{nn}$$ We may also write $\mbox {tr}(A) =\sum _{i=1}^n a_{ii}$.

Let $A=\begin {bmatrix} a _{ij}\end {bmatrix}$ be an $m\times n$ matrix. Then the transpose of $A$, denoted by $A^{T}$, is the $n\times m$ matrix given by switching the rows and columns: \begin{equation*} A^{T} = \begin{bmatrix} a _{ij}\end{bmatrix}^{T}= \begin{bmatrix} a_{ji} \end{bmatrix} \end{equation*}

Let $\vec {v}=\begin {bmatrix}v_1\\v_2\\\vdots \\v_n\end {bmatrix}$ be a direction vector for line $l$ in $\RR ^n$, and let $(a_1, a_2,\ldots , a_n)$ be an arbitrary point on $l$. Then the following vector equation describes $l$: $$\vec {x}(t)=\begin {bmatrix}v_1\\v_2\\\vdots \\v_n\end {bmatrix}t+\begin {bmatrix}a_1\\a_2\\\vdots \\a_n\end {bmatrix}$$

Let $V$ be a nonempty set. Suppose that elements of $V$ can be added together and multiplied by scalars. The set $V$, together with operations of addition and scalar multiplication, is called a vector space provided that

• $V$ is closed under addition
• $V$ is closed under scalar multiplication

and the following properties hold for $\vec {u}$, $\vec {v}$ and $\vec {w}$ in $V$ and scalars $k$ and $p$:

(a)

Commutative Property of Addition: $\vec {u}+\vec {v}=\vec {v}+\vec {u}$

(b)

Associative Property of Addition: $(\vec {u}+\vec {v})+\vec {w}=\vec {u}+(\vec {v}+\vec {w})$

(c)

Existence of Additive Identity: $\vec {u}+\vec {0}=\vec {u}$

(d)

Existence of Additive Inverse: $\vec {u}+(-\vec {u})=\vec {0}$

(e)

Distributive Property over Vector Addition: $k(\vec {u}+\vec {v})=k\vec {u}+k\vec {v}$

(f)

Distributive Property over Scalar Addition: $(k+p)\vec {u}=k\vec {u}+p\vec {u}$

(g)

Associative Property for Scalar Multiplication: $k(p\vec {u})=(kp)\vec {u}$

(h)

Multiplication by $1$: $1\vec {u}=\vec {u}$

We will refer to elements of $V$ as vectors.

The $m\times n$ zero matrix is the $m\times n$ matrix having every entry equal to zero. The zero matrix is denoted by $O$.

The zero transformation, $Z$, maps every element of the domain to the zero vector.

In other words, $$Z:V\rightarrow W$$ is a transformation such that $$Z(\vec {v})=\vec {0}\quad \text {for all}\quad \vec {v} \in V$$