Inner Product Spaces

We have used the dot product in to compute the length of vectors (Corollary cor:length_via_dotprod) and also the angle between vectors. The goal of this section is to define an inner product on an arbitrary vector space over the real numbers. The dot product is an example an inner product for .

A real vector space with an inner product will be called an inner product space. Note that every subspace of an inner product space is again an inner product space using the same inner product.

The next example is important in analysis.

If is any vector, then, using Property prop:inner_prod_3 of Definition def:innerproductspace, we get

and it follows that the number must be zero. This observation is recorded for reference in the following theorem, along with several other properties of inner products. The other proofs are left as Practice Problem ex:10_1_20.

If is an inner product on a space , then, given , , and in ,

for all and in by Property prop:inner_prod_3 and Property prop:inner_prod_4 of Definition def:innerproductspace. Moreover, there is nothing special about the fact that there are two terms in the linear combination or that it is in the first component: and hold for all and in and all , , , and in . These results are described by saying that inner products “preserve” linear combinations. For example,

If is a symmetric matrix and and are columns in , we regard the matrix as a number. If we write

then Properties prop:inner_prod_1 -prop:inner_prod_4 of Definition def:innerproductspace follow from matrix arithmetic (only Property prop:inner_prod_2 of Definition def:innerproductspace requires that is symmetric). Property prop:inner_prod_5 of Definition def:innerproductspace reads and this condition characterizes the positive definite matrices (Theorem thm:024830). This proves the first assertion in the next theorem.

Proof
Given an inner product on , let be the standard basis of . If and are two vectors in , compute by adding the inner product of each term to each term . The result is a double sum. As the reader can verify, this is a matrix product: Hence , where is the matrix whose -entry is . The fact that shows that is symmetric. Finally, is positive definite by Theorem thm:024830.

Thus, just as every linear operator corresponds to an matrix, every inner product on corresponds to a positive definite matrix. In particular, the dot product corresponds to the identity matrix .

Let be an inner product on given as in Theorem thm:030372 by a positive definite matrix . If , then is an expression in the variables called a quadratic form. For more on quadratic forms, see Section 8.8 of [Nicholson], pp. 472–482.

Norm and Distance

Note that Property prop:inner_prod_5 of Definition def:innerproductspace guarantees that , so is a real number.

A vector in an inner product space is called a unit vector if . The set of all unit vectors in is called the unit ball in . For example, if (with the dot product) and , then

Hence the unit ball in is the unit circle with centre at the origin and radius . However, the shape of the unit ball varies with the choice of inner product.

Example exa:030469 graphically illustrates the fact that norms and distances in an inner product space vary with the choice of inner product in .

The next theorem reveals an important and useful fact about the relationship between norms and inner products, extending the Cauchy inequality for (Theorem th:CS).

Proof of Cauchy-Schwarz Inequality
Write and . Using Theorem thm:030346 we compute: It follows that and , and hence that . But then , as desired.

Conversely, if then . Hence (eq:thm10_1_4) shows that or . It follows that one of and is a scalar multiple of the other, even if or .

Another famous inequality, the so-called triangle inequality (See Triangle Inequality in the Appendix), also comes from the Cauchy-Schwarz inequality. It is included in the following list of basic properties of the norm of a vector.

Proof
Because , properties thm:030504a and thm:030504b follow immediately from thm:030346c and thm:030346d of Theorem thm:030346. As to thm:030504c, compute Hence thm:030504c follows by taking positive square roots. Finally, the fact that by the Cauchy-Schwarz inequality gives Hence thm:030504d follows by taking positive square roots.

It is worth noting that the usual triangle inequality for absolute values,

is a special case of thm:030504d where and the dot product is used.

In many calculations in an inner product space, it is required to show that some vector is zero. This is often accomplished most easily by showing that its norm is zero. Here is an example.

The norm properties in Theorem thm:030504 translate to the following properties of distance familiar from geometry. The proof is Practice Problem ex:10_1_21.

Practice Problems

In each case, determine which of Property prop:inner_prod_1–Property prop:inner_prod_5 in Definition def:innerproductspace fail to hold.
(a)
,
(b)
,

Click the arrow to see the answer.

Property prop:inner_prod_5 fails.

(c)
, , where is complex conjugation

Click the arrow to see the answer.

Property prop:inner_prod_1 fails, as sometimes we get a complex number. However, we will return to this definition of in Complex Matrices – see Definition def:025549

(d)
,

Click the arrow to see the answer.

Property prop:inner_prod_5 fails.

(e)
,
(f)
,

Click the arrow to see the answer.

Property prop:inner_prod_5 fails.

Let be an inner product space. If is a subspace, show that is an inner product space using the same inner product.
Property prop:inner_prod_1–Property prop:inner_prod_5 hold in because they hold in .
In each case, find a scalar multiple of that is a unit vector.
(a)
in where
(b)
in where

Click the arrow to see the answer.

(c)
in where
(d)
in ,

Click the arrow to see the answer.

In each case, find the distance between and .
(a)
,
(b)
,

Click the arrow to see the answer.

(c)
, in where and ;
(d)
, in where and ;

Click the arrow to see the answer.

Let be positive numbers. Given and , define . Show that this is an inner product on .
If is a basis of and if and are vectors in , define Show that this is an inner product on .
Let denote the real part of the complex number . Show that is an inner product on if .
If is an isomorphism of the inner product space , show that defines a new inner product on .
Show that every inner product on has the form for some upper triangular matrix with positive diagonal entries.
Theorem thm:024907
In each case, show that defines an inner product on and hence show that is positive definite.
(a)
(b)

Click the arrow to see the answer.

(c)
(d)

Click the arrow to see the answer.

In each case, find a symmetric matrix such that .
(a)
(b)

Click the arrow to see the answer.

(c)
(d)

Click the arrow to see the answer.

If is symmetric and for all columns in , show that .
Consider where .

Click the arrow to see the answer.

By the condition, for all , . Let denote column of . If , then for all and .

Show that the sum of two inner products on is again an inner product.
Let , , , , and . Compute:
(a)
(b)

Given the data in Practice Problem ex:10_1_16, show that .
Show that no vectors exist such that , , and .
Complete Example exa:030310.
Prove Theorem thm:030346.
(a)
Using Property prop:inner_prod_2: .
(b)
Using Property prop:inner_prod_2 and Property prop:inner_prod_4: .
(c)
Using Property prop:inner_prod_3: , so . The rest is Property prop:inner_prod_2.
(d)
Assume that . If this contradicts Property prop:inner_prod_5, so . Conversely, if , then by Part 3 of this theorem.
Prove Theorem thm:030346.
Let and be vectors in an inner product space .
(a)
Expand .
(b)
Expand .

Click the arrow to see the answer.

(c)
Show that .
(d)
Show that .

Click the arrow to see the answer.

Show that for any and in an inner product space.
Let be an inner product on a vector space . Show that the corresponding distance function is translation invariant. That is, show that
for all , , and in .
(a)
Show that for all , in an inner product space .
(b)
If and are two inner products on that have equal associated norm functions, show that holds for all and .
Let denote a vector in an inner product space .
(a)
Show that is a subspace of .
(b)
Let be as in (a). If with the dot product, and if , find a basis for .

Click the arrow to see the answer.

Given vectors and , assume that for each . Show that for all in .
If and holds for each . Show that .
for each , so by Practice Problem ex:10_1_27.
Use the Cauchy-Schwarz inequality in an inner product space to show that:
(a)
If , then for all in .
(b)
for all real , , and .
If in (with the dot product) then . Use (a) with .
(c)
for all vectors , and all in .
If is a matrix, let and denote the rows of .
(a)
Show that .
(b)
Show that .
(a)
If and are nonzero vectors in an inner product space , show that , and hence that a unique angle exists such that
and . This angle is called the angle between and .
(b)
Find the angle between and in with the dot product.
(c)
If is the angle between and , show that the law of cosines is valid:
If , define .
(a)
Show that satisfies the conditions in Theorem thm:030504.
(b)
Show that does not arise from an inner product on given by a matrix .
If it did, use Theorem thm:030372 to find numbers , , and such that for all and .

Text Source

This section was adapted from Section 10.1 of Keith Nicholson’s Linear Algebra with Applications. (CC-BY-NC-SA)

W. Keith Nicholson, Linear Algebra with Applications, Lyryx 2018, Open Edition, pp. 521–530.