Given any linear system $A\vec {x}=\vec {b}$, the system $A\vec {x}=\vec {0}$ is called the associated homogeneous system.

A set $\mathcal {S}$ of vectors is called a basis of $\RR ^n$ (or a basis of a subspace $V$ of $\RR ^n$) provided that

(a)

$\mbox {span}(\mathcal {S})=\RR ^n$ (or $V$)

(b)

$\mathcal {S}$ is linearly independent.

Matrix $A$ of Theorem th:matlintransgeneral is called the matrix of $T$ with respect to ordered bases $\mathcal {B}$ and $\mathcal {C}$.

The equation $$\mbox {det}(A-\lambda I) = 0$$ is called the characteristic equation of $A$.

A set $V$ is said to be closed under addition if for each element $\vec {u} \in V$ and $\vec {v} \in V$ the sum $\vec {u}+\vec {v}$ is also in $V$.

A set $V$ is said to be closed under scalar multiplication if for each element $\vec {v} \in V$ and for each scalar $k \in \RR$ the product $k\vec {v}$ is also in $V$.

Let $A$ be an $m\times n$ matrix. The column space of $A$, denoted by $\mbox {col}(A)$, is the subspace of $\RR ^m$ spanned by the columns of $A$.

Let $U$, $V$ and $W$ be vector spaces, and let $T:U\rightarrow V$ and $S:V\rightarrow W$ be linear transformations. The composition of $S$ and $T$ is the transformation $S\circ T:U\rightarrow W$ given by $$(S\circ T)(\vec {u})=S(T(\vec {u}))$$

Let $\vec {u=\begin {bmatrix}u_1\\u_2\\u_3\end {bmatrix}}$ and $\vec {v}=\begin {bmatrix}v_1\\v_2\\v_3\end {bmatrix}$ be vectors in $\RR ^3$. The cross product of $\vec {u}$ and $\vec {v}$, denoted by $\vec {u}\times \vec {v}$, is given by $$\vec {u}\times \vec {v}=(u_2v_3-u_3v_2)\vec {i}-(u_1v_3-u_3v_1)\vec {j}+(u_1v_2-u_2v_1)\vec {k} =\begin {bmatrix}u_2v_3-u_3v_2\\-u_1v_3+u_3v_1\\u_1v_2-u_2v_1\end {bmatrix}$$

A $2\times 2$ determinant is a number associated with a $2\times 2$ matrix

$$\det {\begin {bmatrix} a & b\\ c & d \end {bmatrix}}=\begin {vmatrix} a & b\\ c & d \end {vmatrix} =ad-bc$$

A $3\times 3$ determinant is a number associated with a $3\times 3$ matrix $$\det {\begin {bmatrix} a_1 & a_2 & a_3\\ b_1 & b_2 &b_3\\ c_1 &c_2 &c_3 \end {bmatrix}}= \begin {vmatrix} a_1 & a_2 & a_3\\ b_1 & b_2 &b_3\\ c_1 &c_2 &c_3 \end {vmatrix} =a_1 \begin {vmatrix} b_2 & b_3\\ c_2 & c_3 \end {vmatrix} -a_2 \begin {vmatrix} b_1 & b_3\\ c_1 & c_3 \end {vmatrix} +a_3 \begin {vmatrix} b_1 & b_2\\ c_1 & c_2 \end {vmatrix}$$

Let $A$ be an $n\times n$ matrix. Then $A$ is said to be diagonalizable if there exists an invertible matrix $P$ such that

Let $V$ be a subspace of $\RR ^n$. The dimension of $V$ is the number, $m$, of elements in any basis of $V$. We write $$\mbox {dim}(V)=m$$

Let $A(a_1, a_2,\ldots ,a_n)$ and $B(b_1, b_2,\ldots ,b_n)$ be points in $\RR ^n$. The distance between $A$ and $B$ is given by $$AB=\sqrt {(a_1-b_1)^2+(a_2-b_2)^2+\ldots +(a_n-b_n)^2}$$

An eigenvalue $\lambda$ of an $n \times n$ matrix $A$ is called a dominant eigenvalue if $\lambda$ has multiplicity $1$, and

Let $\vec {u}$ and $\vec {v}$ be vectors in $\RR ^n$. The dot product of $\vec {u}$ and $\vec {v}$, denoted by $\vec {u}\dotp \vec {v}$, is given by $$\vec {u}\dotp \vec {v}=\begin {bmatrix}u_1\\u_2\\\vdots \\u_n\end {bmatrix}\dotp \begin {bmatrix}v_1\\v_2\\\vdots \\v_n\end {bmatrix}=u_1v_1+u_2v_2+\ldots +u_nv_n$$

The set of all eigenvectors associated with a given eigenvalue of a matrix is known as the eigenspace associated with that eigenvalue.

Let $A$ be an $n \times n$ matrix. We say that a non-zero vector $\vec {x}$ is an eigenvector of $A$ if $$A\vec {x} = \lambda \vec {x}$$ for some scalar $\lambda$. We say that $\lambda$ is an eigenvalue of $A$ associated with the eigenvector $\vec {x}$.

If we are able to diagonalize $A$, say $A=PDP^{-1}$, we say that $PDP^{-1}$ is an eigenvalue decomposition of $A$.

Let $A$ be an $n \times n$ matrix. We say that a non-zero vector $\vec {x}$ is an eigenvector of $A$ if $$A\vec {x} = \lambda \vec {x}$$ for some scalar $\lambda$. We say that $\lambda$ is an eigenvalue of $A$ associated with the eigenvector $\vec {x}$.

An elementary matrix is a square matrix formed by applying a single elementary row operation to the identity matrix.

The following three operations performed on a linear system are called elementary row operations.

(a)

Switching the order of equations (rows) $i$ and $j$: $$R_i\leftrightarrow R_j$$

(b)

Multiplying both sides of equation (row) $i$ by the same non-zero constant, $k$, and replacing equation $i$ with the result: $$kR_i\rightarrow R_i$$

(c)

Adding $k$ times equation (row) $i$ to equation (row) $j$, and replacing equation $j$ with the result: $$R_j+kR_i\rightarrow R_j$$

Two systems of linear equations are said to be equivalent if they have the same solution set.

The process of using the elementary row operations on a matrix to transform it into row-echelon form is called Gaussian Elimination.

The process of using the elementary row operations on a matrix to transform it into reduced row-echelon form is called Gauss-Jordan elimination.

The geometric multiplicity of an eigenvalue $\lambda$ is the dimension of the corresponding eigenspace $\mathcal {S}_\lambda$.

A system of linear equations is called homogeneous if the system can be written in the form $$\begin {array}{ccccccccc} a_{11}x_1 &+ &a_{12}x_2&+&\ldots &+&a_{1n}x_n&= &0 \\ a_{21}x_1 &+ &a_{22}x_2&+&\ldots &+&a_{2n}x_n&= &0 \\ &&&&\vdots &&&& \\ a_{m1}x_1 &+ &a_{m2}x_2&+&\ldots &+&a_{mn}x_n&= &0 \end {array}$$

The identity transformation on $V$, denoted by $\id _V$, is a transformation that maps each element of $V$ to itself.

In other words, $$\id _V:V\rightarrow V$$ is a transformation such that $$\id _V(\vec {v})=\vec {v}\quad \text {for all}\quad \vec {v} \in V$$

Let $V$ and $W$ be vector spaces, and let $T:V\rightarrow W$ be a linear transformation. The image of $T$, denoted by $\mbox {im}(T)$, is the set $$\mbox {im}(T)=\{T(\vec {v}):\vec {v}\in V\}$$ In other words, the image of $T$ consists of individual images of all vectors of $V$.

An inner product on a real vector space $V$ is a function that assigns a real number $\langle \vec {v}, \vec {w}\rangle$ to every pair $\vec {v}$, $\vec {w}$ of vectors in $V$ in such a way that the following properties are satisfied.

(a)

$\langle \vec {v}, \vec {w}\rangle$ is a real number for all $\vec {v}$ and $\vec {w}$ in $V$.

(b)

$\langle \vec {v}, \vec {w}\rangle = \langle \vec {w}, \vec {v}\rangle$ for all $\vec {v}$ and $\vec {w}$ in $V$.

(c)

$\langle \vec {v} + \vec {w}, \vec {u}\rangle = \langle \vec {v}, \vec {u}\rangle + \langle \vec {w}, \vec {u}\rangle$ for all $\vec {u}$, $\vec {v}$, and $\vec {w}$ in $V$.

(d)

$\langle r\vec {v}, \vec {w}\rangle = r\langle \vec {v}, \vec {w}\rangle$ for all $\vec {v}$ and $\vec {w}$ in $V$ and all $r$ in $\RR$.

(e)

$\langle \vec {v}, \vec {v}\rangle > 0$ for all $\vec {v} \neq \vec {0}$ in $V$.

Let $V$ and $W$ be vector spaces, and let $T:V\rightarrow W$ be a linear transformation. A transformation $S:W\rightarrow V$ that satisfies $S\circ T=\id _V$ and $T\circ S=\id _W$ is called an inverse of $T$. If $T$ has an inverse, $T$ is called invertible.

Let $A$ be an $n\times n$ matrix. An $n\times n$ matrix $B$ is called an inverse of $A$ if $$AB=BA=I$$ where $I$ is an $n\times n$ identity matrix. If such an inverse matrix exists, we say that $A$ is invertible. If an inverse does not exist, we say that $A$ is not invertible.

Let $V$ and $W$ be vector spaces. If there exists an invertible linear transformation $T:V\rightarrow W$ we say that $V$ and $W$ are isomorphic and write $V\cong W$. The invertible linear transformation $T$ is called an isomorphism.

Let $V$ and $W$ be vector spaces, and let $T:V\rightarrow W$ be a linear transformation. The kernel of $T$, denoted by $\mbox {ker}(T)$, is the set $$\mbox {ker}(T)=\{\vec {v}:T(\vec {v})=\vec {0}\}$$ In other words, the kernel of $T$ consists of all vectors of $V$ that map to $\vec {0}$ in $W$.

The first non-zero entry in a row of a matrix (when read from left to right) is called the leading entry. When the leading entry is 1, we refer to it as a leading 1.

A vector $\vec {v}$ is said to be a linear combination of vectors $\vec {v}_1, \vec {v}_2,\ldots , \vec {v}_n$ if $$\vec {v}=a_1\vec {v}_1+ a_2\vec {v}_2+\ldots + a_n\vec {v}_n$$ for some scalars $a_1, a_2, \ldots ,a_n$.

A linear equation in variables $x_1, \ldots , x_n$ is an equation that can be written in the form $$a_1x_1+a_2x_2+\ldots +a_nx_n=b$$ where $a_1,\ldots ,a_n$ and $b$ are constants.

A transformation $T:\RR ^n\rightarrow \RR ^m$ is called a linear transformation if the following are true for all vectors $\vec {u}$ and $\vec {v}$ in $\RR ^n$, and scalars $k$.

Let $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k$ be vectors of $\RR ^n$. We say that the set $\{\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k\}$ is linearly independent if the only solution to

If, in addition to the trivial solution, a non-trivial solution (not all $c_1, c_2,\ldots ,c_k$ are zero) exists, then we say that the set $\{\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k\}$ is linearly dependent.

Let $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k$ be vectors of $\RR ^n$. We say that the set $\{\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k\}$ is linearly independent if the only solution to

If, in addition to the trivial solution, a non-trivial solution (not all $c_1, c_2,\ldots ,c_k$ are zero) exists, then we say that the set $\{\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_k\}$ is linearly dependent.

Let $\vec {v}=\begin {bmatrix}v_1\\ v_2\\ \vdots \\v_n\end {bmatrix}$ be a vector in $\RR ^n$, then the length, or the magnitude, of $\vec {v}$ is given by $$\norm {\vec {v}}=\sqrt {v_1^2+v_2^2+\ldots +v_n^2}$$

Let $A=\begin {bmatrix} a_{ij}\end {bmatrix}$ and $B=\begin {bmatrix} b_{ij}\end {bmatrix}$ be two $m\times n$ matrices. Then the sum of matrices $A$ and $B$, denoted by $A+B$, is an $m \times n$ matrix given by $$A+B=\begin {bmatrix}a_{ij}+b_{ij}\end {bmatrix}$$

Let $A=\begin {bmatrix} a_{ij}\end {bmatrix}$ and $B=\begin {bmatrix} b_{ij}\end {bmatrix}$ be two $m \times n$ matrices. Then $A=B$ means that $a_{ij}=b_{ij}$ for all $1\leq i\leq m$ and $1\leq j\leq n$.

Let $A$ be an $m\times n$ matrix whose rows are vectors $\vec {r}_1$, $\vec {r}_2,\ldots ,\vec {r}_n$. Let $B$ be an $n\times p$ matrix with columns $\vec {b}_1, \vec {b}_2, \ldots , \vec {b}_p$. Then the entries of the matrix product $AB$ are given by the dot products $$AB=\begin {bmatrix}-&\vec {r}_1&-\\-&\vec {r}_2&-\\ &\vdots & \\-&\vec {r}_i &-\\ &\vdots & \\-&\vec {r}_m&-\end {bmatrix}\begin {bmatrix}|&|&&|&&|\\\vec {b}_1& \vec {b}_2 &\ldots & \vec {b}_j&\ldots & \vec {b}_p\\|&|&&|&&|\end {bmatrix}=$$ $$=\begin {bmatrix}\vec {r}_1\dotp \vec {b}_1&\vec {r}_1\dotp \vec {b}_2&\ldots &\vec {r}_1\dotp \vec {b}_j&\ldots &\vec {r}_1\dotp \vec {b}_p\\\vec {r}_2\dotp \vec {b}_1&\vec {r}_2\dotp \vec {b}_2&\ldots &\vec {r}_2\dotp \vec {b}_j&\ldots &\vec {r}_2\dotp \vec {b}_p\\\vdots &\vdots &&\vdots &&\vdots \\\vec {r}_i\dotp \vec {b}_1&\vec {r}_i\dotp \vec {b}_2&\ldots &\vec {r}_i\dotp \vec {b}_j&\ldots &\vec {r}_i\dotp \vec {b}_p\\\vdots &\vdots &&\vdots &&\vdots \\\vec {r}_m\dotp \vec {b}_1&\vec {r}_m\dotp \vec {b}_2&\ldots &\vec {r}_m\dotp \vec {b}_j&\ldots &\vec {r}_m\dotp \vec {b}_p\end {bmatrix}$$

If $A=\begin {bmatrix} a_{ij}\end {bmatrix}$ and $k$ is a scalar, then $kA=\begin {bmatrix} ka_{ij}\end {bmatrix}$.

Let $A$ be an $m\times n$ matrix, and let $\vec {x}$ be an $n\times 1$ vector. The product $A\vec {x}$ is the $m\times 1$ vector given by: $$A\vec {x}=\begin {bmatrix} a_{11} & a_{12}&\dots &a_{1n}\\ a_{21}&a_{22} &\dots &a_{2n}\\ \vdots & \vdots &\ddots &\vdots \\ a_{m1}&\dots &\dots &a_{mn} \end {bmatrix}\begin {bmatrix}x_1\\x_2\\\vdots \\x_n\end {bmatrix}= x_1\begin {bmatrix}a_{11}\\a_{21}\\ \vdots \\a_{m1}\end {bmatrix}+ x_2\begin {bmatrix}a_{12}\\a_{22}\\ \vdots \\a_{m2}\end {bmatrix}+\dots + x_n\begin {bmatrix}a_{1n}\\a_{2n}\\ \vdots \\a_{mn}\end {bmatrix}$$ or, equivalently, $$A\vec {x}=\begin {bmatrix}a_{11}x_1+a_{12}x_2+\ldots +a_{1n}x_n\\a_{21}x_1+a_{22}x_2+\ldots +a_{2n}x_n\\\vdots \\a_{m1}x_1+a_{m2}x_2+\ldots +a_{mn}x_n\end {bmatrix}$$

Matrix $A$ of Theorem th:matlintransgeneral is called the matrix of $T$ with respect to ordered bases $\mathcal {B}$ and $\mathcal {C}$.

A square matrix $A$ is said to be nonsingular provided that $\mbox {rref}(A)=I$. Otherwise we say that $A$ is singular.

Let $A$ be an $m\times n$ matrix. The null space of $A$, denoted by $\mbox {null}(A)$, is the set of all vectors $\vec {x}$ in $\RR ^n$ such that $A\vec {x}=\vec {0}$.

The nullity of a linear transformation $T:V\rightarrow W$, is the dimension of the kernel of $T$. $$\mbox {nullity}(T)=\mbox {dim}(\mbox {ker}(T))$$

Let $A$ be a matrix. The dimension of the null space of $A$ is called the nullity of $A$. $$\mbox {dim}\Big (\mbox {null}(A)\Big )=\mbox {nullity}(A)$$

A linear transformation $T:V\rightarrow W$ is one-to-one if $$T(\vec {v}_1)=T(\vec {v}_2)\quad \text {implies that}\quad \vec {v}_1=\vec {v}_2$$

A linear transformation $T:V\rightarrow W$ is onto if for every element $\vec {w}$ of $W$, there exists an element $\vec {v}$ of $V$ such that $T(\vec {v})=\vec {w}$.

If $W$ is a subspace of $\RR ^n$, define the orthogonal complement $W^\perp$ of $W$ (pronounced “$W$-perp”) by

Let $W$ be a subspace of $\RR ^n$ and let $\vec {x} \in \RR ^n$. Then there exist unique vectors $\vec {w} \in W$ and $\vec {w}^\perp \in W^\perp$ such that $\vec {x} = \vec {w} + \vec {w}^\perp$.

An $n \times n$ matrix $Q$ is called an orthogonal matrix if it satisfies one (and hence all) of the conditions in Theorem th:orthogonal_matrices.

Let $W$ be a subspace of $\RR ^n$ with orthogonal basis $\{\vec {f}_{1}, \vec {f}_{2}, \dots , \vec {f}_{m}\}$. If $\vec {x}$ is in $\RR ^n$, the vector

Let $\vec {v}$ be a vector, and let $\vec {d}$ be a non-zero vector. The projection of $\vec {v}$ onto $\vec {d}$ is given by $$\mbox {proj}_{\vec {d}}\vec {v}=\left (\frac {\vec {v}\dotp \vec {d}}{\norm {\vec {d}}^2}\right )\vec {d}$$

Let $\{ \vec {v}_1, \vec {v}_2, \cdots , \vec {v}_k \}$ be a set of nonzero vectors in $\RR ^n$. Then this set is called an orthogonal set if $\vec {v}_i \dotp \vec {v}_j = 0$ for all $i \neq j$. Moreover, if $\norm {\vec {v}_i}=1$ for $i=1,\ldots ,m$ (i.e. each vector in the set is a unit vector), we say the set of vectors is an orthonormal set.

Let $\vec {u}$ and $\vec {v}$ be vectors in $\RR ^n$. We say $\vec {u}$ and $\vec {v}$ are orthogonal if $\vec {u}\dotp \vec {v}=0$.

An $n \times n$ matrix $A$ is said to be orthogonally diagonalizable if an orthogonal matrix $Q$ can be found such that $Q^{-1}AQ = Q^{T}AQ$ is diagonal.

Let $\{ \vec {v}_1, \vec {v}_2, \cdots , \vec {v}_k \}$ be a set of nonzero vectors in $\RR ^n$. Then this set is called an orthogonal set if $\vec {v}_i \dotp \vec {v}_j = 0$ for all $i \neq j$. Moreover, if $\norm {\vec {v}_i}=1$ for $i=1,\ldots ,m$ (i.e. each vector in the set is a unit vector), we say the set of vectors is an orthonormal set.

Let $\vec {v}=\begin {bmatrix}v_1\\v_2\\\vdots \\v_n\end {bmatrix}$ be a direction vector for line $l$ in $\RR ^n$, and let $(a_1, a_2,\ldots , a_n)$ be an arbitrary point on $l$. Then the following parametric equations describe $l$: $$x_1=v_1t+a_1$$ $$x_2=v_2t+a_2$$ $$\vdots$$ $$x_n=v_nt+a_n$$

A square matrix is called positive definite if it is symmetric and all its eigenvalues $\lambda$ are positive. We write $\lambda >0$ when eigenvalues are real and positive.

Let $A$ be an $m \times n$ matrix with independent columns. A QR-factorization of $A$ expresses it as $A = QR$ where $Q$ is $m \times n$ with orthonormal columns and $R$ is an invertible and upper triangular matrix with positive diagonal entries.

The rank of a linear transformation $T:V\rightarrow W$, is the dimension of the image of $T$. $$\mbox {rank}(T)=\mbox {dim}(\mbox {im}(T))$$

The rank of matrix $A$, denoted by $\mbox {rank}(A)$, is the number of nonzero rows that remain after we reduce $A$ to row-echelon form by elementary row operations.

For any matrix $A$, $$\mbox {rank}(A)=\mbox {dim}\Big (\mbox {row}(A)\Big )$$

Let $T:V\rightarrow W$ be a linear transformation. Suppose $\mbox {dim}(V)=n$, then $$\mbox {rank}(T)+\mbox {nullity}(T)=n$$

Let $A$ be an $m\times n$ matrix. Then $$\mbox {rank}(A)+\mbox {nullity}(A)=n$$

A matrix that is already in row-echelon form is said to be in reduced row-echelon form if:

(a)

Each leading entry is $1$

(b)

All entries above and below each leading $1$ are $0$

Let $\{\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\}$ be a set of vectors in $\RR ^n$. If we can remove one vector without changing the span of this set, then that vector is redundant. In other words, if $$\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\right )=\mbox {span}\left (\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_{j-1},\vec {v}_{j+1},\dots ,\vec {v}_k\right )$$ we say that $\vec {v}_j$ is a redundant element of $\{\vec {v}_1,\vec {v}_2,\dots ,\vec {v}_k\}$, or simply redundant.

A matrix is said to be in row-echelon form if:

(a)

All entries below each leading entry are 0.

(b)

Each leading entry is in a column to the right of the leading entries in the rows above it.

(c)

All rows of zeros, if there are any, are located below non-zero rows.

Let $A$ be an $m\times n$ matrix. The row space of $A$, denoted by $\mbox {row}(A)$, is the subspace of $\RR ^n$ spanned by the rows of $A$.

If $A$ and $B$ are $n \times n$ matrices, we say that $A$ and $B$ are similar, if $B = P^{-1}AP$ for some invertible matrix $P$. In this case we write $A \sim B$.

A square matrix $A$ is said to be nonsingular provided that $\mbox {rref}(A)=I$. Otherwise we say that $A$ is singular.

Let $A$ be an $m\times n$ matrix. The singular values of $A$ are the square roots of the positive eigenvalues of $A^TA.$

An $n\times n$ matrix $A$ is said to be symmetric if $A=A^{T}.$ It is said to be skew symmetric if $A=-A^{T}.$

Let $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$ be vectors in $\RR ^n$. The set $S$ of all linear combinations of $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$ is called the span of $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$. We write $$S=\mbox {span}(\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p)$$ and we say that vectors $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$ span $S$. Any vector in $S$ is said to be in the span of $\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p$. The set $\{\vec {v}_1, \vec {v}_2,\ldots ,\vec {v}_p\}$ is called a spanning set for $S$.

If we are able to diagonalize $A$, say $A=PDP^{-1}$, we say that $PDP^{-1}$ is an eigenvalue decomposition of $A$.

The set $\{\vec {e}_1, \ldots ,\vec {e}_n\}$ is called the standard basis of $\RR ^n$.

The matrix in Theorem th:matlin is known as the standard matrix of the linear transformation $T$.

Let $\vec {e}_i$ denote a vector that has $1$ as the $i^{th}$ component and zeros elsewhere. In other words, $$\vec {e}_i=\begin {bmatrix} 0\\ 0\\ \vdots \\ 1\\ \vdots \\ 0 \end {bmatrix}$$ where $1$ is in the $i^{th}$ position. We say that $\vec {e}_i$ is a standard unit vector of $\RR ^n$.

Let $A=[a_{ij}]$ be the $n\times n$ matrix which is the coefficient matrix of the linear system $A \vec {x}= \vec {b}$. Let $$r_i(A):= \sum _{j \ne i} |a_{ij}|$$ denote the sum of the absolute values of the non-diagonal entries in row $i$. We say that $A$ is strictly diagonally dominant if $$|a_{ii}|>r_i(A)$$ for all values of $i$ from $i=1$ to $i=n$.

A nonempty subset $U$ of a vector space $V$ is called a subspace of $V$, provided that $U$ is itself a vector space when given the same addition and scalar multiplication as $V$.

Suppose that $V$ is a nonempty subset of $\RR ^n$ that is closed under addition and closed under scalar multiplication. Then $V$ is a subspace of $\RR ^n$.

Let $U$ be a nonempty subset of a vector space $V$. If $U$ is closed under the operations of addition and scalar multiplication of $V$, then $U$ is a subspace of $V$.

An $n\times n$ matrix $A$ is said to be symmetric if $A=A^{T}.$ It is said to be skew symmetric if $A=-A^{T}.$

The trace of an $n \times n$ matrix $A$, abbreviated $\mbox {tr} A$, is defined to be the sum of the main diagonal elements of $A$. In other words, if $A = [a_{ij}]$, then $$\mbox {tr}(A) = a_{11} + a_{22} + \dots + a_{nn}$$ We may also write $\mbox {tr}(A) =\sum _{i=1}^n a_{ii}$.

Let $A=\begin {bmatrix} a _{ij}\end {bmatrix}$ be an $m\times n$ matrix. Then the transpose of $A$, denoted by $A^{T}$, is the $n\times m$ matrix given by

Let $\vec {v}=\begin {bmatrix}v_1\\v_2\\\vdots \\v_n\end {bmatrix}$ be a direction vector for line $l$ in $\RR ^n$, and let $(a_1, a_2,\ldots , a_n)$ be an arbitrary point on $l$. Then the following vector equation describes $l$: $$\vec {x}(t)=\begin {bmatrix}v_1\\v_2\\\vdots \\v_n\end {bmatrix}t+\begin {bmatrix}a_1\\a_2\\\vdots \\a_n\end {bmatrix}$$

Let $V$ be a nonempty set. Suppose that elements of $V$ can be added together and multiplied by scalars. The set $V$, together with operations of addition and scalar multiplication, is called a vector space provided that

• $V$ is closed under addition
• $V$ is closed under scalar multiplication

and the following properties hold for $\vec {u}$, $\vec {v}$ and $\vec {w}$ in $V$ and scalars $k$ and $p$:

(a)

Commutative Property of Addition: $\vec {u}+\vec {v}=\vec {v}+\vec {u}$

(b)

Associative Property of Addition: $(\vec {u}+\vec {v})+\vec {w}=\vec {u}+(\vec {v}+\vec {w})$

(c)

Existence of Additive Identity: $\vec {u}+\vec {0}=\vec {u}$

(d)

Existence of Additive Inverse: $\vec {u}+(-\vec {u})=\vec {0}$

(e)

Distributive Property over Vector Addition: $k(\vec {u}+\vec {v})=k\vec {u}+k\vec {v}$

(f)

Distributive Property over Scalar Addition: $(k+p)\vec {u}=k\vec {u}+p\vec {u}$

(g)

Associative Property for Scalar Multiplication: $k(p\vec {u})=(kp)\vec {u}$

(h)

Multiplication by $1$: $1\vec {u}=\vec {u}$

We will refer to elements of $V$ as vectors.

The $m\times n$ zero matrix is the $m\times n$ matrix having every entry equal to zero. The zero matrix is denoted by $O$.

The zero transformation, $Z$, maps every element of the domain to the zero vector.

In other words, $$Z:V\rightarrow W$$ is a transformation such that $$Z(\vec {v})=\vec {0}\quad \text {for all}\quad \vec {v} \in V$$