Linear Independence

If a friend told you that they have a line spanned by and and , you would probably think that your friend’s description is a little excessive. Isn’t one of the above vectors sufficient to describe the line? A line can be described as a span of one vector, but it can also be described as a span of two or more vectors. There are many advantages, however, to using the most efficient description possible. In this section we will begin to explore what makes a description “more efficient.”

Redundant Vectors

Consider the following collection of vectors: What is the span of these vectors? A line, , A parallelogram, A parallelepiped

In this Exploration we will examine what can happen to the span of a collection of vectors when a vector is removed from the collection.

First, let’s remove from .

Which of the following is true?

is a line is a parallelogram.

Removing from changed, did not change the span.

Now let’s remove from the original collection of vectors.

Which of the following is true?

is a line is the right side of the coordinate plane. is a parallelogram.

Removing from changed, did not change the span.

As you just discovered, removing a vector from a collection of vectors may or may not affect the span of the collection. We will refer to vectors that can be removed from a collection without changing the span as redundant. In Exploration exp:redundantVecs1, is redundant, while is not.

Our next goal is to see what causes of Exploration exp:redundantVecs1 to be redundant. The answer lies not in the vector itself, but in its relationship to the other vectors in the collection. Observe that . In other words, is a scalar multiple of another vector in the set. To see why this matters, let’s pick an arbitrary vector in . Vector is in the span because it can be written as a linear combination of the three vectors as follows

But is not essential to this linear combination because it can be replaced with , as shown below.

Regardless of what vector we write as a linear combination of, and , we will always be able to replace with , placing into the span of and , and making redundant. (Note that we can just as easily write , and argue that is redundant.) We conclude that only one of and is needed to maintain the span of the original three vectors. We have The left-most collection in this expression contains redundant vectors; the other two collections do not.

In Exploration exp:redundantVecs1 we found one vector to be redundant because we could replace it with a scalar multiple of another vector in the set. The following Exploration delves into what happens when a vector in a given set is a linear combination of the other vectors.

Consider the set of vectors The three vectors are shown below. RIGHT-CLICK and DRAG to rotate the interactive graph.

is A line, A plane, , A parallelepiped

Can we remove one of the vectors from the set without changing the span? Observe that we can write as a linear combination of the other two vectors \begin{equation} \label{eq:redundant}\begin{bmatrix}4\\4\\-1\end{bmatrix}=2\begin{bmatrix}1\\2\\-1\end{bmatrix}+1\begin{bmatrix}2\\0\\1\end{bmatrix} \end{equation}

This means that we can write any vector in as a linear combination of only and by replacing with the expression in (eq:redundant). For example,

We have

We conclude that vector is redundant. Can each of the other two vectors in the set be considered redundant? You will address this question in Practice Problem prob:redundant1.

Collections of vectors that do not contain redundant vectors are very important in linear algebra. We will refer to such collections as linearly independent. Collections of vectors that contain redundant vectors will be called linearly dependent. The following section offers a definition that will allow us to easily determine linear dependence and independence of vectors.

Linear Independence

Proof
th:lindeplincombofother_a th:lindeplincombofother_b If are linearly dependent, then \begin{equation*} c_1\vec{v}_1+c_2\vec{v}_2+\ldots +c_j\vec{v}_j+\ldots +c_k\vec{v}_k=\vec{0} \end{equation*}

has a non-trivial solution. In other words at least one of the constants, say , does not equal zero. This allows us to solve for :

\begin{align*} -c_j\vec{v}_j&=c_1\vec{v}_1+c_2\vec{v}_2+\ldots +c_k\vec{v}_k \\ \\ \vec{v}_j&=-\frac{c_1}{c_j}\vec{v}_1-\frac{c_2}{c_j}\vec{v}_2-\ldots -\frac{c_k}{c_j}\vec{v}_k \end{align*}

(Do you see why it was important to have one of the constants nonzero?) This shows that may be expressed as a linear combination of the other vectors.

th:lindeplincombofother_b th:lindeplincombofother_c First, suppose is a linear combination of . We will show that is redundant by showing that To show equality of the two spans we will pick a vector in the left span and show that it is also an element of the span on the right. Then, we will pick a vector in the right span and show that it is also an element of the span on the left, and we will conclude that the sets are equal.

Observe that if is in , then it has to be in . (Why?)

Now suppose is in . We need to show that is also in .

By assumption, we can write as \begin{equation} \label{eq:vj} \vec{v}_j=a_1\vec{v}_1+a_2\vec{v}_2+\dots +a_{j-1}\vec{v}_{j-1}+a_{j+1}\vec{v}_{j+1}+\dots +a_k\vec{v}_k. \end{equation}

Since is in , we have Substituting the expression in (eq:vj) for and simplifying, we obtain the following

\begin{eqnarray*} \vec{w}=(b_1+b_ja_1)\vec{v}_1+(b_2+b_ja_2)\vec{v}_2&+&\dots \\ &+&(b_{j-1}+b_ja_{j-1})\vec{v}_{j-1}\\ &+&(b_{j+1}+b_ja_{j+1})\vec{v}_{j+1}\\ &+&\dots \\ &+&(b_k+b_ja_k)\vec{v}_k. \end{eqnarray*}
This shows that is in . We now have which shows that is redundant.

th:lindeplincombofother_c th:lindeplincombofother_a Suppose that is redundant, so that Consider a vector in \begin{equation} \label{eq:w1} \vec{w}=a_1\vec{v}_1+a_2\vec{v}_2+\dots +a_j\vec{v}_j+\dots +a_k\vec{v}_k \end{equation} Since the span contains ALL possible linear combinations of , we may choose such that .

By assumption, is also in . Therefore, we can express as a linear combination \begin{equation} \label{eq:w2} \vec{w}=b_1\vec{v}_1+b_2\vec{v}_2+\dots +b_{j-1}\vec{v}_{j-1}+b_{j+1}\vec{v}_{j+1}+\dots +b_k\vec{v}_k. \end{equation}

We complete the proof by showing there exists a non-trivial solution to \begin{equation} \label{eq:LinIndepDefRepeated} c_1\vec{v}_1+c_2\vec{v}_2+\ldots +c_j\vec{v}_j+\ldots +c_k\vec{v}_k=\vec{0}. \end{equation} Subtracting expression (eq:w2) from (eq:w1) we obtain

\begin{eqnarray*} \vec{0}=\vec{w}-\vec{w}=(a_1-b_1)\vec{v}_1&+&\dots \\ \nonumber &+&(a_{j-1}-b_{j-1})\vec{v}_{j-1}+a_j\vec{v}_j+(a_{j+1}-b_{j+1})\vec{v}_{j+1}\\ \nonumber &+&\dots \\ \nonumber &+&(a_k-b_k)\vec{v}_k \end{eqnarray*}
Recall that we ensured that . This implies that we have a non-trivial solution to Equation eq:LinIndepDefRepeated.

These three parts of the proof show that if one of the conditions is true, all three must be true. It is a logical consequence that if one of the three conditions is false, all three must be false.

Geometry of Linearly Dependent and Linearly Independent Vectors

Theorem th:lindeplincombofother gives us a convenient ways of looking at linear dependence/independence geometrically. When looking at two or more vectors, we ask, “can one of the vectors be written as a linear combination of the others?” We can also ask, “is one of the vectors redundant?” If the answer to either of these questions is “YES”, then the vectors are linearly dependent.

A Set of Two Vectors

Two vectors are linearly dependent if and only if one is a scalar multiple of the other. Two nonzero linearly dependent vectors may look like this:

or like this:

Two linearly independent vectors will look like this:

A Set of Three Vectors

Given a set of three nonzero vectors, we have the following possibilities:

  • (Linearly Dependent Vectors) The three vectors are scalar multiples of each other.

  • (Linearly Dependent Vectors) Two of the vectors are scalar multiples of each other.

  • (Linearly Dependent Vectors) One vector can be viewed as the diagonal of a parallelogram determined by scalar multiples of the other two vectors. All three vectors lie in the same plane.

  • (Linearly Independent Vectors) A set of three vectors is linearly independent if the vectors do not lie in the same plane. For example, vectors , and are linearly independent.

Practice Problems

In Exploration exp:redundantVecs2 we considered the following set of vectors

and demonstrated that is redundant by using the fact that it is a linear combination of the other two vectors.

(a)
Express each of and as a linear combination of the remaining vectors.

(b)
Which of the following is NOT true?
If is in , then is in . Both and are redundant in . We can remove and from at the same time without affecting the span.

Problems prob:linindmultchoice1-prob:linindmultchoice4

Are the given vectors linearly independent?

Yes No
If we rewrite as a system of linear equations, there will be more unknowns than equations.
Yes No
If we let be the matrix whose columns are these vectors, then should tell us what we want to know.
Yes No
If we let be the matrix whose columns are these vectors, then should tell us what we want to know.
Yes No
In a set of two vectors, the only way one could be redundant is if they are scalar multiples of each other.

Problems prob:TFlinind1-prob:TFlinind2

True or False?

Any set containing the zero vector is linearly dependent.
TRUE FALSE
Can the zero vector be removed from the set without changing the span?
A set containing five vectors in is linearly dependent.
TRUE FALSE
If we rewrite Equation eq:defLinInd for five vectors in as a system of equations, how many equations and unknowns will it have? What does this imply about the number of solutions?

Problems prob:linindmultchoice5-prob:linindmultchoice7

Each problem below provides information about vectors . If possible, determine whether the vectors are linearly dependent or independent.

The vectors are linearly independent The vectors are linearly dependent There is not enough information given to make a determination
The vectors are linearly independent The vectors are linearly dependent There is not enough information given to make a determination
The vectors are linearly independent The vectors are linearly dependent There is not enough information given to make a determination

Problems prob:linindmultchoice9-prob:linindmultchoice10

Each diagram below shows a collection of vectors. Are the vectors linearly dependent or independent?

The vectors are linearly independent The vectors are linearly dependent There is not enough information given to make a determination

The vectors are linearly independent The vectors are linearly dependent There is not enough information given to make a determination

Suppose is a linearly independent set in , and that is not in . Prove that is also linearly independent.
Suppose is a linearly independent set of vectors. Prove that the set is also linearly independent.
Suppose is a linearly independent set of vectors in . Is the following set dependent or independent ? Prove your claim.
2024-09-26 20:46:30