Inner Product Spaces

We have used the dot product in \(\RR ^n\) to compute the length of vectors (Corollary cor:length_via_dotprod) and also the angle between vectors. The goal of this section is to define an inner product on an arbitrary vector space \(V\) over the real numbers. The dot product is an example an inner product for \(\RR ^n\).

A real vector space \(V\) with an inner product \(\langle \ , \rangle \) will be called an inner product space. Note that every subspace of an inner product space is again an inner product space using the same inner product.

The next example is important in analysis.

If \(\vec {v}\) is any vector, then, using Property prop:inner_prod_3 of Definition def:innerproductspace, we get

\begin{equation*} \langle \vec {0}, \vec {v} \rangle = \langle \vec {0} + \vec {0}, \vec {v} \rangle = \langle \vec {0}, \vec {v} \rangle + \langle \vec {0}, \vec {v} \rangle \end{equation*}

and it follows that the number \(\langle \vec {0}, \vec {v}\rangle \) must be zero. This observation is recorded for reference in the following theorem, along with several other properties of inner products. The other proofs are left as Practice Problem ex:10_1_20.

If \(\langle \ , \rangle \) is an inner product on a space \(V\), then, given \(\vec {u}\), \(\vec {v}\), and \(\vec {w}\) in \(V\),

\begin{equation*} \langle r\vec {u} + s\vec {v}, \vec {w} \rangle = \langle r\vec {u}, \vec {w} \rangle + \langle s\vec {v}, \vec {w} \rangle = r\langle \vec {u}, \vec {w} \rangle + s\langle \vec {v}, \vec {w} \rangle \end{equation*}

for all \(r\) and \(s\) in \(\RR \) by Property prop:inner_prod_3 and Property prop:inner_prod_4 of Definition def:innerproductspace. Moreover, there is nothing special about the fact that there are two terms in the linear combination or that it is in the first component:

\begin{equation*} \langle r_1\vec {v}_1 + r_2\vec {v}_2 + \dots + r_n\vec {v}_n, \vec {w} \rangle = r_1\langle \vec {v}_1, \vec {w} \rangle + r_2\langle \vec {v}_2, \vec {w} \rangle + \dots + r_n\langle \vec {v}_n, \vec {w} \rangle \end{equation*}

and

\begin{equation*} \langle \vec {v}, s_1\vec {w}_1 + s_2\vec {w}_2 + \dots + s_m\vec {w}_m \rangle = s_1\langle \vec {v}, \vec {w}_1 \rangle + s_2\langle \vec {v}, \vec {w}_2 \rangle + \dots + s_m\langle \vec {v}, \vec {w}_m \rangle \end{equation*}

hold for all \(r_{i}\) and \(s_{i}\) in \(\RR \) and all \(\vec {v}\), \(\vec {w}\), \(\vec {v}_{i}\), and \(\vec {w}_{j}\) in \(V\). These results are described by saying that inner products “preserve” linear combinations. For example,

\begin{align*} \langle 2\vec {u} - \vec {v}, 3\vec {u} + 2\vec {v} \rangle &= \langle 2\vec {u}, 3\vec {u} \rangle + \langle 2\vec {u}, 2\vec {v} \rangle + \langle -\vec {v}, 3\vec {u} \rangle + \langle -\vec {v}, 2\vec {v} \rangle \\ &= 6 \langle \vec {u}, \vec {u} \rangle + 4 \langle \vec {u}, \vec {v} \rangle -3 \langle \vec {v}, \vec {u} \rangle - 2 \langle \vec {v}, \vec {v} \rangle \\ &= 6 \langle \vec {u}, \vec {u} \rangle + \langle \vec {u}, \vec {v} \rangle - 2 \langle \vec {v}, \vec {v} \rangle \end{align*}

If \(A\) is a symmetric \(n \times n\) matrix and \(\vec {x}\) and \(\vec {y}\) are columns in \(\RR ^n\), we regard the \(1 \times 1\) matrix \(\vec {x}^{T}A\vec {y}\) as a number. If we write

\begin{equation*} \langle \vec {x}, \vec {y} \rangle = \vec {x}^TA\vec {y} \quad \mbox { for all columns } \vec {x}, \vec {y} \mbox { in } \RR ^n \end{equation*}

then Properties prop:inner_prod_1 -prop:inner_prod_4 of Definition def:innerproductspace follow from matrix arithmetic (only Property prop:inner_prod_2 of Definition def:innerproductspace requires that \(A\) is symmetric). Property prop:inner_prod_5 of Definition def:innerproductspace reads

\begin{equation*} \vec {x}^TA \vec {x} > 0 \quad \mbox { for all columns } \vec {x} \neq \vec {0} \mbox { in } \RR ^n \end{equation*}

and this condition characterizes the positive definite matrices (Theorem thm:024830). This proves the first assertion in the next theorem.

Given an inner product \(\langle \ , \rangle \) on \(\RR ^n\), let \(\{\vec {e}_{1}, \vec {e}_{2}, \dots , \vec {e}_{n}\}\) be the standard basis of \(\RR ^n\). If \(\vec {x} = \displaystyle \sum _{i = 1}^{n} x_i\vec {e}_i\) and \(\vec {y} = \displaystyle \sum _{j = 1}^{n} y_j\vec {e}_j\) are two vectors in \(\RR ^n\), compute \(\langle \vec {x}, \vec {y}\rangle \) by adding the inner product of each term \(x_{i}\vec {e}_{i}\) to each term \(y_{j}\vec {e}_{j}\). The result is a double sum.

\begin{equation*} \langle \vec {x}, \vec {y} \rangle = \displaystyle \sum _{i = 1}^{n} \sum _{j = 1}^{n} \langle x_i \vec {e}_i, y_j\vec {e}_j \rangle = \displaystyle \sum _{i = 1}^{n} \sum _{j = 1}^{n} x_i \langle \vec {e}_i, \vec {e}_j \rangle y_j \end{equation*}
As the reader can verify, this is a matrix product:
\begin{equation*} \langle \vec {x}, \vec {y} \rangle = \left [ \begin{array}{cccc} x_1 & x_2 & \cdots & x_n \\ \end{array} \right ] \left [ \begin{array}{cccc} \langle \vec {e}_1, \vec {e}_1 \rangle & \langle \vec {e}_1, \vec {e}_2 \rangle & \cdots & \langle \vec {e}_1, \vec {e}_n \rangle \\ \langle \vec {e}_2, \vec {e}_1 \rangle & \langle \vec {e}_2, \vec {e}_2 \rangle & \cdots & \langle \vec {e}_2, \vec {e}_n \rangle \\ \vdots & \vdots & \ddots & \vdots \\ \langle \vec {e}_n, \vec {e}_1 \rangle & \langle \vec {e}_n, \vec {e}_2 \rangle & \cdots & \langle \vec {e}_n, \vec {e}_n \rangle \\ \end{array} \right ] \left [ \begin{array}{c} y_1 \\ y_2 \\ \vdots \\ y_n \end{array} \right ] \end{equation*}
Hence \(\langle \vec {x}, \vec {y}\rangle = \vec {x}^{T}A\vec {y}\), where \(A\) is the \(n \times n\) matrix whose \((i, j)\)-entry is \(\langle \vec {e}_{i}, \vec {e}_{j} \rangle \). The fact that
\begin{equation*} \langle \vec {e}_{i}, \vec {e}_{j}\rangle = \langle \vec {e}_{j}, \vec {e}_{i}\rangle \end{equation*}
shows that \(A\) is symmetric. Finally, \(A\) is positive definite by Theorem thm:024830.

Thus, just as every linear operator \(\RR ^n \to \RR ^n\) corresponds to an \(n \times n\) matrix, every inner product on \(\RR ^n\) corresponds to a positive definite \(n \times n\) matrix. In particular, the dot product corresponds to the identity matrix \(I_{n}\).

Let \(\langle \ , \rangle \) be an inner product on \(\RR ^n\) given as in Theorem thm:030372 by a positive definite matrix \(A\). If \(\vec {x} = \left [ \begin{array}{cccc} x_1 & x_2 & \cdots & x_n \end{array} \right ]^T \), then \(\langle \vec {x}, \vec {x}\rangle = \vec {x}^{T}A\vec {x}\) is an expression in the variables \(x_{1}, x_{2}, \dots , x_{n}\) called a quadratic form. For more on quadratic forms, see Section 8.8 of [Nicholson], pp. 472–482.

Norm and Distance

Note that Property (e) of Definition 1 guarantees that \(\langle \vec {v}, \vec {v}\rangle \geq 0\), so \(\norm {\vec {v}}\) is a real number.

A vector \(\vec {v}\) in an inner product space \(V\) is called a unit vector if \(\norm {\vec {v}} = 1\). The set of all unit vectors in \(V\) is called the unit ball in \(V\). For example, if \(V = \RR ^2\) (with the dot product) and \(\vec {v} = (x, y)\), then

\begin{equation*} \norm { \vec {v} }^2 = 1 \quad \mbox { if and only if } \quad x^2 + y^2 = 1 \end{equation*}

Hence the unit ball in \(\RR ^2\) is the unit circle \(x^{2} + y^{2} = 1\) with centre at the origin and radius \(1\). However, the shape of the unit ball varies with the choice of inner product.

Example 20 graphically illustrates the fact that norms and distances in an inner product space \(V\) vary with the choice of inner product in \(V\).

The next theorem reveals an important and useful fact about the relationship between norms and inner products, extending the Cauchy inequality for \(\RR ^n\) (Theorem ??).

Write \(\norm {\vec {v}} = a\) and \(\norm {\vec {w}} = b\). Using Theorem 8 we compute:

\begin{equation}\label {eq:thm10_1_4} \begin{split} \norm { b\vec {v} - a \vec {w} }^2 &= b^2 \norm { \vec {v} }^2 - 2ab \langle \vec {v}, \vec {w} \rangle + a^2\norm { \vec {w} }^2 = 2ab(ab - \langle \vec {v}, \vec {w} \rangle ) \\ \norm { b\vec {v} + a \vec {w} }^2 &= b^2 \norm { \vec {v} }^2 + 2ab \langle \vec {v}, \vec {w} \rangle + a^2\norm { \vec {w} }^2 = 2ab(ab + \langle \vec {v}, \vec {w} \rangle ) \\ \end{split} \end{equation}
It follows that \(ab - \langle \vec {v}, \vec {w}\rangle \geq 0\) and \(ab + \langle \vec {v}, \vec {w}\rangle \geq 0\), and hence that \(-ab \leq \langle \vec {v}, \vec {w}\rangle \leq ab\). But then \(| \langle \vec {v}, \vec {w}\rangle | \leq ab = \norm {\vec {v}} \norm { \vec {w}}\), as desired.

Conversely, if \(|\langle \vec {v}, \vec {w}\rangle | = \norm {\vec {v}} \norm { \vec {w} } = ab\) then \(\langle \vec {v}, \vec {w}\rangle = \pm ab\). Hence (1) shows that \(b\vec {v} - a\vec {w} = \vec {0}\) or \(b\vec {v} + a\vec {w} = \vec {0}\). It follows that one of \(\vec {v}\) and \(\vec {w}\) is a scalar multiple of the other, even if \(a = 0\) or \(b = 0\).

Another famous inequality, the so-called triangle inequality (See Triangle Inequality in the Appendix), also comes from the Cauchy-Schwarz inequality. It is included in the following list of basic properties of the norm of a vector.

Because \(\norm { \vec {v} } = \sqrt {\langle \vec {v}, \vec {v} \rangle }\), properties (a) and (b) follow immediately from ?? and ?? of Theorem 8. As to (c), compute

\begin{equation*} \norm { r\vec {v} } ^2 = \langle r\vec {v}, r\vec {v} \rangle = r^2\langle \vec {v}, \vec {v} \rangle = r^2\norm { \vec {v} }^2 \end{equation*}
Hence (c) follows by taking positive square roots. Finally, the fact that \(\langle \vec {v}, \vec {w}\rangle \leq \norm {\vec {v}}\norm {\vec {w}}\) by the Cauchy-Schwarz inequality gives
\begin{align*} \norm { \vec {v} + \vec {w} } ^2 = \langle \vec {v} + \vec {w}, \vec {v} + \vec {w} \rangle &= \norm { \vec {v} } ^2 + 2 \langle \vec {v}, \vec {w} \rangle + \norm { \vec {w} } ^2 \\ &\leq \norm { \vec {v} } ^2 + 2 \norm { \vec {v} } \norm { \vec {w} } + \norm { \vec {w} } ^2 \\ &= (\norm { \vec {v} } + \norm { \vec {w} })^2 \end{align*}

Hence (d) follows by taking positive square roots.

It is worth noting that the usual triangle inequality for absolute values,

\begin{equation*} | r + s | \leq |r| + |s| \mbox { for all real numbers } r \mbox { and } s \end{equation*}

is a special case of (d) where \(V = \RR = \RR ^1\) and the dot product \(\langle r, s \rangle = rs\) is used.

In many calculations in an inner product space, it is required to show that some vector \(\vec {v}\) is zero. This is often accomplished most easily by showing that its norm \(\norm {\vec {v}}\) is zero. Here is an example.

The norm properties in Theorem 26 translate to the following properties of distance familiar from geometry. The proof is Practice Problem .

Practice Problems

In each case, determine which of Property (a)–Property (e) in Definition 1 fail to hold.
(a)
\(V = \RR ^2\), \(\left \langle \begin{bmatrix}x_1\\ y_1\end{bmatrix}, \begin{bmatrix}x_2\\ y_2\end{bmatrix} \right \rangle = x_1y_1x_2y_2\)
(b)
\(V = \RR ^3\),
\(\left \langle \begin{bmatrix}x_1\\ x_2\\ x_3\end{bmatrix}, \begin{bmatrix}y_1\\ y_2\\ y_3\end{bmatrix} \right \rangle = x_1y_1 - x_2y_2 + x_3y_3\)

Click the arrow to see the answer.

Property (e) fails.
(c)
\(V = \mathbb {C}\), \(\langle z, w \rangle = z\overline {w}\), where \(\overline {w}\) is complex conjugation

Click the arrow to see the answer.

Property (a) fails, as sometimes we get a complex number. However, we will return to this definition of \(\langle \ , \rangle \) in Complex Matrices – see Definition ??
(d)
\(V = \mathbb {P}^3\), \(\langle p(x), q(x) \rangle = p(1)q(1)\)

Click the arrow to see the answer.

Property (e) fails.
(e)
\(V = \mathbb {M}_{22}\), \(\langle A, B \rangle = \mbox {det}(AB)\)
(f)
\(V = \mathcal {F}[0, 1]\), \(\langle f, g \rangle = f(1)g(0) + f(0)g(1)\)

Click the arrow to see the answer.

Property (e) fails.
Let \(V\) be an inner product space. If \(U \subseteq V\) is a subspace, show that \(U\) is an inner product space using the same inner product.
Property (a)–Property (e) hold in \(U\) because they hold in \(V\).
In each case, find a scalar multiple of \(\vec {v}\) that is a unit vector.
(a)
\(\vec {v} = f\) in \(\mathcal {C}[0, 1]\) where \(f(x) = x^2\)
\(\langle f, g \rangle \int _{0}^{1} f(x)g(x)dx\)
(b)
\(\vec {v} = f\) in \(\mathcal {C}[-\pi , \pi ]\) where \(f(x) = \cos x\)
\(\langle f, g \rangle \int _{-\pi }^{\pi } f(x)g(x)dx\)

Click the arrow to see the answer.

\(\frac {1}{\sqrt {\pi }}f\)
(c)
\(\vec {v} = \left [ \begin{array}{r} 1 \\ 3 \end{array} \right ]\) in \(\RR ^2\) where \(\langle \vec {v}, \vec {w} \rangle = \vec {v}^T \left [ \begin{array}{rr} 1 & 1 \\ 1 & 2 \end{array} \right ] \vec {w}\)
(d)
\( \vec {v} = \left [ \begin{array}{r} 3 \\ -1 \end{array} \right ]\) in \(\RR ^2\), \(\langle \vec {v}, \vec {w} \rangle = \vec {v}^T \left [ \begin{array}{rr} 1 & -1 \\ -1 & 2 \end{array} \right ] \vec {w}\)

Click the arrow to see the answer.

\(\frac {1}{\sqrt {17}} \left [ \begin{array}{r} 3 \\ -1 \end{array} \right ]\)
In each case, find the distance between \(\vec {u}\) and \(\vec {v}\).
(a)
\(\vec {u} = \begin{bmatrix}3\\ -1\\ 2\\ 0\end{bmatrix}\), \(\vec {v} = \begin{bmatrix}1\\ 1\\ 1\\ 3\end{bmatrix}; \langle \vec {u}, \vec {v} \rangle = \vec {u} \dotp \vec {v}\)
(b)
\(\vec {u} = \begin{bmatrix}1\\ 2\\ -1\\ 2\end{bmatrix}\), \(\vec {v} = \begin{bmatrix}2\\ 1\\ -1\\ 3\end{bmatrix}; \langle \vec {u}, \vec {v} \rangle = \vec {u} \dotp \vec {v}\)

Click the arrow to see the answer.

\(\sqrt {3}\)
(c)
\(\vec {u} = f\), \(\vec {v} = g \) in \(\mathcal {C}[0, 1]\) where \(f(x) = x^2 \) and \(g(x) = 1 - x\); \(\langle f, g \rangle = \int _{0}^{1} f(x)g(x)dx\)
(d)
\(\vec {u} = f\), \(\vec {v} = g \) in \(\mathcal {C}[-\pi , \pi ]\) where \(f(x) = 1\) and \(g(x) = \cos x\); \(\langle f, g \rangle = \int _{-\pi }^{\pi } f(x)g(x)dx\)

Click the arrow to see the answer.

\(\sqrt {3\pi }\)
Let \(a_{1}, a_{2}, \dots , a_{n}\) be positive numbers. Given \(\vec {v} = \begin{bmatrix}v_{1}\\ v_{2}\\ \vdots \\ v_{n}\end{bmatrix}\) and \(\vec {w} = \begin{bmatrix}w_{1}\\ w_{2}\\ \vdots \\ w_{n}\end{bmatrix}\), define \(\langle \vec {v}, \vec {w}\rangle = a_{1}v_{1}w_{1} + \dots + a_{n}v_{n}w_{n}\). Show that this is an inner product on \(\RR ^n\).
If \(\{\vec {b}_{1}, \dots , \vec {b}_{n}\}\) is a basis of \(V\) and if \(\vec {v} = v_1\vec {b}_1 + \dots + v_n\vec {b}_n\) and \(\vec {w} = w_1\vec {b}_1 + \dots + w_n\vec {b}_n\) are vectors in \(V\), define
\begin{equation*} \langle \vec {v}, \vec {w} \rangle = v_1w_1 + \dots + v_nw_n . \end{equation*}
Show that this is an inner product on \(V\).
Let \(\mbox {re}(z)\) denote the real part of the complex number \(z\). Show that \(\langle \ , \rangle \) is an inner product on \(\mathbb {C}\) if \(\langle \vec {z}, \vec {w}\rangle = \mbox {re}(z\overline {w})\).
If \(T : V \to V\) is an isomorphism of the inner product space \(V\), show that
\begin{equation*} \langle \vec {v}, \vec {w} \rangle _1 = \langle T(\vec {v}), T(\vec {w}) \rangle \end{equation*}
defines a new inner product \(\langle \ , \rangle _{1}\) on \(V\).
Show that every inner product \(\langle \ , \rangle \) on \(\RR ^n\) has the form \(\langle \vec {x}, \vec {y}\rangle = (U\vec {x}) \dotp (U\vec {y})\) for some upper triangular matrix \(U\) with positive diagonal entries.
Theorem ??
In each case, show that \(\langle \vec {v}, \vec {w}\rangle = \vec {v}^{T}A\vec {w}\) defines an inner product on \(\RR ^2\) and hence show that \(A\) is positive definite.
(a)
\(A = \left [ \begin{array}{rr} 2 & 1 \\ 1 & 1 \end{array} \right ]\)
(b)
\(A = \left [ \begin{array}{rr} 5 & -3 \\ -3 & 2 \end{array} \right ]\)

Click the arrow to see the answer.

\( \langle \vec {v}, \vec {v} \rangle = 5v_1^2 - 6v_1v_2 + 2v_2^2 = \frac {1}{5}[(5v_1 - 3v_2)^2 + v_2^2] \)
(c)
\(A = \left [ \begin{array}{rr} 3 & 2 \\ 2 & 3 \end{array} \right ]\)
(d)
\(A = \left [ \begin{array}{rr} 3 & 4 \\ 4 & 6 \end{array} \right ]\)

Click the arrow to see the answer.

\( \langle \vec {v}, \vec {v} \rangle = 3v_1^2 + 8v_1v_2 + 6v_2^2 = \frac {1}{3}[(3v_1 + 4v_2)^2 + 2v_2^2] \)
In each case, find a symmetric matrix \(A\) such that \(\langle \vec {v}, \vec {w}\rangle = \vec {v}^{T}A\vec {w}\).
(a)
\( \left \langle \left [ \begin{array}{r} v_1 \\ v_2 \end{array} \right ], \left [ \begin{array}{r} w_1 \\ w_2 \end{array} \right ] \right \rangle = v_1w_1 + 2v_1w_2 + 2v_2w_1 + 5v_2w_2\)
(b)
\(\left \langle \left [ \begin{array}{r} v_1 \\ v_2 \end{array} \right ], \left [ \begin{array}{r} w_1 \\ w_2 \end{array} \right ] \right \rangle = v_1w_1 - v_1w_2 - v_2w_1 + 2v_2w_2\)

Click the arrow to see the answer.

\(\left [ \begin{array}{rr} 1 & -2 \\ -2 & 1 \end{array} \right ]\)
(c)
\(\left \langle \left [ \begin{array}{r} v_1 \\ v_2 \\ v_3 \end{array} \right ], \left [ \begin{array}{r} w_1 \\ w_2 \\ w_3 \end{array} \right ] \right \rangle = 2v_1w_1 + v_2w_2 + v_3w_3 - v_1w_2 \\ -v_2w_1 + v_2w_3 + v_3w_2\)
(d)
\(\left \langle \left [ \begin{array}{r} v_1 \\ v_2 \\ v_3 \end{array} \right ], \left [ \begin{array}{r} w_1 \\ w_2 \\ w_3 \end{array} \right ] \right \rangle = v_1w_1 + 2v_2w_2 + 5v_3w_3 \\ - 2v_1w_3 - 2v_3w_1\)

Click the arrow to see the answer.

\(\left [ \begin{array}{rrr} 1 & 0 & -2 \\ 0 & 2 & 0 \\ -2 & 0 & 5 \end{array} \right ]\)
If \(A\) is symmetric and \(\vec {x}^{T}A\vec {x} = 0\) for all columns \(\vec {x}\) in \(\RR ^n\), show that \(A = 0\).
Consider \(\langle \vec {x} + \vec {y}, \vec {x} + \vec {y} \rangle \) where \(\langle \vec {x}, \vec {y} \rangle = \vec {x}^TA\vec {y}\).

Click the arrow to see the answer.

By the condition, \(\langle \vec {x}, \vec {y} \rangle = \frac {1}{2} \langle \vec {x} + \vec {y}, \vec {x} + \vec {y} \rangle = 0\) for all \(\vec {x}\), \(\vec {y}\). Let \(\vec {e}_{i}\) denote column \(i\) of \(I\). If \(A = \left [ a_{ij} \right ]\), then \(a_{ij} = \vec {e}_{i}^{T}A\vec {e}_{j} = \{\vec {e}_{i}, \vec {e}_{j}\} = 0\) for all \(i\) and \(j\).
Show that the sum of two inner products on \(V\) is again an inner product.
Let \( \norm { \vec {u} } = 1\), \(\norm { \vec {v} } = 2\), \(\norm { \vec {w} } = \sqrt {3} \), \(\langle \vec {u}, \vec {v} \rangle = -1\), \(\langle \vec {u}, \vec {w}\rangle = 0\) and \(\langle \vec {v}, \vec {w}\rangle = 3\). Compute:
(a)
\(\langle \vec {v} + \vec {w}, 2\vec {u} - \vec {v} \rangle \)
(b)
\(\langle \vec {u} - 2 \vec {v} - \vec {w}, 3\vec {w} - \vec {v} \rangle \)

\(\answer {-15}\)

Given the data in Practice Problem , show that \(\vec {u} + \vec {v} = \vec {w}\).
Show that no vectors exist such that \(\norm {\vec {u}} = 1\), \(\norm {\vec {v}} = 2\), and \(\langle \vec {u}, \vec {v}\rangle = -3\).
Complete Example 3.
Prove Theorem 8.
(a)
Using Property (b): \( \langle \vec {u}, \vec {v} + \vec {w} \rangle = \langle \vec {v} + \vec {w}, \vec {u} \rangle = \langle \vec {v}, \vec {u} \rangle + \langle \vec {w}, \vec {u} \rangle = \langle \vec {u}, \vec {v} \rangle + \langle \vec {u}, \vec {w} \rangle \).
(b)
Using Property (b) and Property (d): \(\langle \vec {v}, r\vec {w} \rangle = \langle r\vec {w}, \vec {v} \rangle = r \langle \vec {w}, \vec {v} \rangle = r \langle \vec {v}, \vec {w} \rangle \).
(c)
Using Property (c): \(\langle \vec {0}, \vec {v} \rangle = \langle \vec {0} + \vec {0}, \vec {v} \rangle = \langle \vec {0}, \vec {v} \rangle + \langle \vec {0}, \vec {v} \rangle \), so \( \langle \vec {0}, \vec {v} \rangle = 0\). The rest is Property (b).
(d)
Assume that \(\langle \vec {v}, \vec {v} \rangle = 0\). If \(\vec {v} \neq \vec {0}\) this contradicts Property (e), so \(\vec {v} = \vec {0}\). Conversely, if \(\vec {v} = \vec {0}\), then \(\langle \vec {v}, \vec {v} \rangle = 0\) by Part 3 of this theorem.
Prove Theorem 8.
Let \(\vec {u}\) and \(\vec {v}\) be vectors in an inner product space \(V\).
(a)
Expand \(\langle 2\vec {u} - 7\vec {v}, 3\vec {u} + 5\vec {v} \rangle \).
(b)
Expand \(\langle 3\vec {u} - 4\vec {v}, 5\vec {u} + \vec {v} \rangle \).

Click the arrow to see the answer.

\(15\norm {\vec {u}}^{2} - 17 \langle \vec {u}, \vec {v} \rangle - 4\norm {\vec {v}}^{2}\)
(c)
Show that \(\norm { \vec {u} + \vec {v} } ^2 = \norm { \vec {u} } ^2 + 2 \langle \vec {u}, \vec {v} \rangle + \norm { \vec {v} } ^2 \).
(d)
Show that \(\norm { \vec {u} - \vec {v} } ^2 = \norm { \vec {u} } ^2 - 2 \langle \vec {u}, \vec {v} \rangle + \norm { \vec {v} } ^2\).

Click the arrow to see the answer.

\(\norm {\vec {u} + \vec {v}}^{2} = \langle \vec {u} + \vec {v}, \vec {u} + \vec {v} \rangle = \norm {\vec {u}}^{2} + 2\langle \vec {u}, \vec {v}\rangle + \norm {\vec {v}}^{2}\)
Show that
\begin{equation*} \norm { \vec {v} } ^2 + \norm { \vec {w} } ^2 = \frac {1}{2} \{ \norm { \vec {v} + \vec {w} } ^2 + \norm { \vec {v} - \vec {w} } ^2\} \end{equation*}
for any \(\vec {v}\) and \(\vec {w}\) in an inner product space.
Let \(\langle \ , \rangle \) be an inner product on a vector space \(V\). Show that the corresponding distance function is translation invariant. That is, show that
\(\mbox {d}(\vec {v}, \vec {w}) = \mbox {d}(\vec {v} + \vec {u}, \vec {w} + \vec {u})\) for all \(\vec {v}\), \(\vec {w}\), and \(\vec {u}\) in \(V\).
(a)
Show that \(\langle \vec {u}, \vec {v} \rangle = \frac {1}{4}[\norm { \vec {u} + \vec {v} } ^2 - \norm { \vec {u} - \vec {v} } ^2]\) for all \(\vec {u}\), \(\vec {v}\) in an inner product space \(V\).
(b)
If \(\langle \ , \rangle \) and \(\langle \ , \rangle ^\prime \) are two inner products on \(V\) that have equal associated norm functions, show that \(\langle \vec {u}, \vec {v}\rangle = \langle \vec {u}, \vec {v}\rangle ^\prime \) holds for all \(\vec {u}\) and \(\vec {v}\).
Let \(\vec {v}\) denote a vector in an inner product space \(V\).
(a)
Show that \(W = \{\vec {w} \mid \vec {w} \mbox { in } V, \langle \vec {v}, \vec {w} = 0\}\) is a subspace of \(V\).
(b)
Let \(W\) be as in (a). If \(V = \RR ^3\) with the dot product, and if \(\vec {v} = \begin{bmatrix}1\\ -1\\ 2\end{bmatrix}\), find a basis for \(W\).

Click the arrow to see the answer.

\(\left \{\begin{bmatrix}1\\ 1\\ 0\end{bmatrix}, \begin{bmatrix}0\\ 2\\ 1\end{bmatrix}\right \}\)
Given vectors \(\vec {w}_{1}, \vec {w}_{2}, \dots , \vec {w}_{n}\) and \(\vec {v}\), assume that \(\langle \vec {v}, \vec {w}_{i}\rangle = 0\) for each \(i\). Show that \(\langle \vec {v}, \vec {w}\rangle = 0\) for all \(\vec {w}\) in \(\mbox {span}\{\vec {w}_{1}, \vec {w}_{2}, \dots , \vec {w}_{n}\}\).
If \(V = \mbox {span}\{\vec {v}_{1}, \vec {v}_{2}, \dots , \vec {v}_{n}\}\) and \(\langle \vec {v}, \vec {v}_{i}\rangle = \langle \vec {w}, \vec {v}_i\rangle \) holds for each \(i\). Show that \(\vec {v} = \vec {w}\).
\(\langle \vec {v} - \vec {w}, \vec {v}_{i} \rangle = \langle \vec {v}, \vec {v}_{i} \rangle - \langle \vec {w}, \vec {v}_{i} \rangle = 0\) for each \(i\), so \(\vec {v} = \vec {w}\) by Practice Problem .
Use the Cauchy-Schwarz inequality in an inner product space to show that:
(a)
If \(\norm {\vec {u}} \leq 1\), then \(\langle \vec {u}, \vec {v}\rangle ^{2} \leq \norm {\vec {v}}^{2}\) for all \(\vec {v}\) in \(V\).
(b)
\((x \cos \theta + y \sin \theta )^{2} \leq x^{2} + y^{2}\) for all real \(x\), \(y\), and \(\theta \).
If \(\vec {u} = (\cos \theta , \sin \theta )\) in \(\RR ^2\) (with the dot product) then \(\norm {\vec {u}} = 1\). Use (a) with \(\vec {v} = \begin{bmatrix}x\\ y\end{bmatrix}\).
(c)
\(\norm { r_1\vec {v}_1 + \dots + r_n\vec {v}_n } ^2 \leq [r_1 \norm { \vec {v}_1 } + \dots + r_n \norm { \vec {v}_n } ]^2\) for all vectors \(\vec {v}_{i}\), and all \(r_{i} > 0\) in \(\RR \).
If \(A\) is a \(2 \times n\) matrix, let \(\vec {u}\) and \(\vec {v}\) denote the rows of \(A\).
(a)
Show that \(AA^T = \left [ \begin{array}{rr} \norm { \vec {u} } ^2 & \vec {u} \dotp \vec {v} \\ \vec {u} \dotp \vec {v} & \norm { \vec {v} } ^2 \end{array} \right ]\).
(b)
Show that \(\mbox {det}(AA^{T}) \geq 0\).
(a)
If \(\vec {v}\) and \(\vec {w}\) are nonzero vectors in an inner product space \(V\), show that \(-1 \leq \frac {\langle \vec {v}, \vec {w} \rangle }{\norm { \vec {v} } \norm { \vec {w} }} \leq 1\), and hence that a unique angle \(\theta \) exists such that
\(\frac {\langle \vec {v}, \vec {w} \rangle }{\norm { \vec {v} } \norm { \vec {w} }} = \cos \theta \) and \(0 \leq \theta \leq \pi \). This angle \(\theta \) is called the angle between \(\vec {v}\) and \(\vec {w}\).
(b)
Find the angle between \(\vec {v} = \begin{bmatrix}1\\ 2\\ -1\\ 1\\ 3\end{bmatrix}\) and \(\vec {w} = \begin{bmatrix}2\\ 1\\ 0\\ 2\\ 0\end{bmatrix}\) in \(\RR ^5\) with the dot product.
(c)
If \(\theta \) is the angle between \(\vec {v}\) and \(\vec {w}\), show that the law of cosines is valid:
\begin{equation*} \norm { \vec {v} - \vec {w} } = \norm { \vec {v} } ^2 + \norm { \vec {w} } ^2 - 2\norm { \vec {v} } \norm { \vec {w} } \cos \theta . \end{equation*}
If \(V = \RR ^2\), define \(\norm {\begin{bmatrix}x\\ y\end{bmatrix}} = |x| + |y|\).
(a)
Show that \(\norm {\dotp }\) satisfies the conditions in Theorem 26.
(b)
Show that \(\norm {\dotp }\) does not arise from an inner product on \(\RR ^2\) given by a matrix \(A\).
If it did, use Theorem 10 to find numbers \(a\), \(b\), and \(c\) such that \(\norm {\begin{bmatrix}x\\ y\end{bmatrix}}^{2} = ax^{2} + bxy + cy^{2}\) for all \(x\) and \(y\).

Text Source

This section was adapted from Section 10.1 of Keith Nicholson’s Linear Algebra with Applications. (CC-BY-NC-SA)

W. Keith Nicholson, Linear Algebra with Applications, Lyryx 2018, Open Edition, pp. 521–530.