Pre-Class Warmup
Example
Solve the linear system
\[\left\{ \begin{array}\ 2x_1 &+& x_2 &=& 0 \\ x_1 &-& 3x_2 &=& 7 \\ &-& 2x_2 &=& 4 \end{array} \right. \qquad \longleftrightarrow \qquad \left( \begin{matrix} 2 & 1 \\ 1 & -3 \\ 0 & -2 \end{matrix} \; \left| \; \begin{matrix} 0 \\ 7 \\ 4 \end{matrix} \right. \right)\]
by using Gauss-Jordan elimination.
Today we are starting Chapter 2 of the book by discussing vectors, linear combinations, and span.
Vectors
If you took Math 126, you have already learned vector arithmetic, but let's summarize quickly.
Definition
A vector is an ordered list of real numbers \(\mathbf{v}=(v_1, v_2, \dots, v_n).\) They can be written horizontally as row vectors, as in the previous sentence, or vertically as column vectors,
\(\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}.\)
(Actually, we will more commonly think of vectors as column vectors.) The numbers \(v_i\) are called the components of \(\mathbf{v}\).
For a given positive integer \(n\), the set of all vectors with \(n\) components is called \(n\)-dimensional Euclidean space, and is denoted \(\mathbb{R}^n\) (pronounced "R-n"). In particular, \(\mathbb{R}^2\) is called the Euclidean plane.
Vectors can be added together and multiplied by real numbers (in this context, we refer to real numbers as scalars, since they "scale" the components). Recall that both of these operations are performed component-wise:
\[\begin{pmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{pmatrix} + \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix} = \begin{pmatrix} u_1+v_1 \\ u_2+v_2 \\ \vdots \\ u_n+v_n \end{pmatrix} \qquad \text{and} \qquad c \cdot \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix} = \begin{pmatrix} cv_1 \\ cv_2 \\ \vdots \\ cv_n \end{pmatrix}.\]
(Notice that addition is only defined for vectors with the same number of components.) The zero vector in \(\mathbb{R}^n\), simply denoted \(\mathbf{0}\), is the vector consisting of \(n\) zeros. The zero vector is the additive identity for \(\mathbb{R}^n\), meaning that \(\mathbf{v} + \mathbf{0} = \mathbf{v}\) and \(\mathbf{0} + \mathbf{v} = \mathbf{v}\) for any \(\mathbf{v} \in \mathbb{R}^n\).
Vector addition and scalar multiplication have many of the properties of regular addition and multiplication, including commutativity, associativity, and distributivity. See Theorem 2.3 in your book if you want the full list.
Two words of caution:
- In some sources, vectors are written in bold to help distinguish them from regular numbers. This isn't easy to do on paper or at the board, so often people will use "vector hats", like \(\vec{v}\), when writing vectors. Often I will forget to do either, but it should be clear from context whether something is a vector or a scalar.
- The "zero vector" in \(\mathbb{R}^2\) (for example) is not the same object as the "zero vector" in \(\mathbb{R}^3\), even though they both have all zeros as components and are both written as \(\mathbf{0}\). (They "live in different worlds," namely the 2-dimensional plane and 3-dimensional space, so it makes no sense to compare them.)
Vectors, geometrically
We usually draw vectors as arrows, going from the origin (the tail of the vector) to the point in \(n\)-dimensional space corresponding to the vector (the head of the vector). In this way, we can visualize addition and scalar multiplication of vectors geometrically:
- To add two vectors \(u\) and \(v\) geometrically, translate \(v\) so that its tail lines up with the head of \(u\). Then \(u+v\) goes from the origin to the head of the translated vector, \(v\). (You can instead translate \(u\) to line up with \(v\), and you will get the same result.) Equivalently, you can form a parallelogram by translating both vectors; then \(u+v\) is the diagonal of this parallelogram from the origin. (This is called the parallelogram rule. See the below image from your book.)
- Multiplying a vector \(v\) by a scalar \(c\) essentially "scales" \(v\) to make it \(c\) times as long. However, if \(c\) is negative, then the vector gets "flipped" around the origin:
- To subtract two vectors geometrically, say \(u-v\), we can think of this as adding \(u\) to \(-v\) (that is, \(\left(-1\right)v\)), and then simply apply the previous two rules.
Linear combinations of vectors
The most important operation we can perform on a group of vectors is taking a linear combination of them:
Definition
Let \(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m\) be vectors in \(\mathbb{R}^n\). A linear combination of \(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m\) is a vector of the form
\[x_1 \mathbf{v}_1 + x_2 \mathbf{v}_2 + \dotsm + x_m \mathbf{v}_m\]
where \(x_1, x_2, \dots, x_m\) are scalars, sometimes called the coefficients of the linear combination.
In other words, to get a linear combination of a list of vectors, we scale each of them by any amount and then add the results together.
Example
Let \(\mathbf{u} = \begin{pmatrix} 2 \\ 1 \\ 0 \end{pmatrix}\), \(\mathbf{v} = \begin{pmatrix} 1 \\ -3 \\ -2 \end{pmatrix}\), and \(\mathbf{w} = \begin{pmatrix} 0 \\ 7 \\ 4 \end{pmatrix}\) be vectors in \(\mathbb{R}^3\).
You can check that \(\mathbf{u} - 2 \mathbf{v} = \mathbf{w}\). (No, really, check it for yourself!) Therefore, \(\mathbf{w}\) is a linear combination of \(\mathbf{u}\) and \(\mathbf{v}\).
Notice that the scalars in a linear combination can be zero: that is, you don't need to "use" all the vectors to get a linear combination. For example:
- Given any vectors \(\mathbf{u}\) and \(\mathbf{v}\), it is always the case that \(\mathbf{u}\) is a linear combination of \(\mathbf{u}\) and \(\mathbf{v}\), because
\[\mathbf{u} = 1 \mathbf{u} + 0 \mathbf{v}.\] -
The zero vector \(\mathbf{0}\) is always a linear combination of any list of vectors \(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m\), because
\(\mathbf{0} = 0\mathbf{v}_1 + 0\mathbf{v}_2 + \dotsm + 0\mathbf{v}_m.\)
(The bold \(\mathbf{0}\) on the left-hand side is a vector, and all the \(0\)'s on the right-hand side are scalars.)
Recognizing linear combinations
A natural question at this point is: if I have some list of vectors \(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m\) and another vector \(\mathbf{w}\), how can I tell whether \(\mathbf{w}\) is a linear combination of \(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m\)? Let's consider an example.
Example
Consider the vectors \(\mathbf{u} = \begin{pmatrix} 2 \\ 1 \\ 0 \end{pmatrix}\), \(\mathbf{v} = \begin{pmatrix} 1 \\ -3 \\ -2 \end{pmatrix}\), and \(\mathbf{w} = \begin{pmatrix} 0 \\ 7 \\ 4 \end{pmatrix}\) from the previous example.
Suppose we wanted to figure out a priori whether \(\mathbf{w}\) is a linear combination of \(\mathbf{u}\) and \(\mathbf{v}\). To do so, we refer to the definition: \(\mathbf{w}\) is a linear combination of \(\mathbf{u}\) and \(\mathbf{v}\) if and only if there are real numbers \(x_1\) and \(x_2\) such that \(x_1\mathbf{u} + x_2\mathbf{v} = \mathbf{w}\). Writing this out, we have
\[x_1 \begin{pmatrix} 2 \\ 1 \\ 0 \end{pmatrix} + x_2 \begin{pmatrix} 1 \\ -3 \\ -2 \end{pmatrix} = \begin{pmatrix} 0 \\ 7 \\ 4 \end{pmatrix}.\]
We can manipulate the left-hand side using the rules of scalar multiplication and vector addition, which gives us
\[\begin{pmatrix} 2x_1 + x_2 \\ x_1 - 3x_2 \\ -2x_2 \end{pmatrix} = \begin{pmatrix} 0 \\ 7 \\ 4 \end{pmatrix}.\]
Now this is saying that two vectors are equal, which means that all corresponding components have to be equal. So, we end up with a list of linear equations, which all have to be true at the same time: a system of linear equations! In our case, we get the system (and corresponding augmented matrix)
\[\left\{ \begin{array}\ 2x_1 &+& x_2 &=& 0 \\ x_1 &-& 3x_2 &=& 7 \\ &-& 2x_2 &=& 4 \end{array} \right. \qquad \longleftrightarrow \qquad \left( \begin{matrix} 2 & 1 \\ 1 & -3 \\ 0 & -2 \end{matrix} \; \left| \; \begin{matrix} 0 \\ 7 \\ 4 \end{matrix} \right. \right).\]
(This is the system from the pre-class warmup.) We should get the (unique) solution \((x_1, x_2) = (1, -2)\), which corresponds to the fact that \(\mathbf{u} - 2 \mathbf{v} = \mathbf{w}\) (as we knew from the earlier example).
In general, if we want to determine whether one vector is a linear combination of a list of vectors, then to do so, we can set up and solve the right linear system, like in the above example. It will be a linear combination if and only if that system has at least one solution (that is, the system is consistent).
Which linear system? Well, in the above example, notice that the vectors we started with ended up as the columns of the augmented matrix, with \(\mathbf{u}\) and \(\mathbf{v}\) on the left of the vertical line, and \(\mathbf{w}\) on the right. In general, we can follow the same steps as above, and the augmented matrix will always be related in the same way to the vectors we started with. This gives us the following theorem:
Theorem
Let \(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m\) and \(\mathbf{w}\) be vectors in \(\mathbb{R}^n\). Then \(\mathbf{w}\) is a linear combination of \(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m\) if and only if the linear system given by the augmented matrix
\[\left( \begin{matrix} | & | & & | \\ \mathbf{v}_1 & \mathbf{v}_2 & \dotsm & \mathbf{v}_m \\ | & | & & | \end{matrix} \; \left| \; \begin{matrix} | \\ \mathbf{w} \\ | \end{matrix} \right. \right)\]
is consistent. (This notation means that we put \(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m\) and \(\mathbf{w}\) as the columns of the augmented matrix.)
Span of vectors
To talk about linear combinations more easily, we introduce the following concept:
Definition
Let \(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m\) be vectors in \(\mathbb{R}^n\). Their span, written
\[\text{span}(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m),\]
is the subset of \(\mathbb{R}^n\) consisting of all linear combinations of these vectors.
Thus, the question "is \(\mathbf{w}\) a linear combination of \(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m\)?" is equivalent to the question "is \(\mathbf{w}\) in \(\text{span}(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m)\)?"
Span, geometrically
We can now gain some intuition about linear combinations by thinking about what \(\text{span}(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_m)\) looks like geometrically.
Imagine we are in \(\mathbb{R}^2\) (the plane).
- Let's start with a single nonzero vector \(\mathbf{v}\). (A vector being "nonzero" means that not all components are zero, so \(\mathbf{v}\) could still have some zero components.) Linear combinations of one vector \(\mathbf{v}\) are just the same as scalar multiples of \(\mathbf{v}\), which we know are the result of "stretching" \(\mathbf{v}\) by some amount, and possibly flipping it (if we multiply by a negative number). So, geometrically, the span of \(\mathbf{v}\) is the line passing through the origin that "contains" \(\mathbf{v}\). We say that \(\mathbf{v}\) "spans" that line.
- Suppose now we include some other (nonzero) vector \(\mathbf{w}\) in our span. Then, by definition, \(\text{span}(\mathbf{v}, \mathbf{w})\) is the set of all linear combinations of \(\mathbf{v}\) and \(\mathbf{w}\); in other words, the set of all sums of a multiple of \(\mathbf{v}\) and a multiple of \(\mathbf{w}\). There are two cases to consider:
- First, we could choose \(\mathbf{w}\) to be on the same line that "contains" \(\mathbf{v}\). In that case, all multiples of \(\mathbf{v}\) and \(\mathbf{w}\) will be on that line, so their sum will also be on the line. In other words, \(\text{span}(\mathbf{v}, \mathbf{w})\) is the same as \(\text{span}(\mathbf{v})\). In a certain sense, \(\mathbf{w}\) is "redundant" in this scenario.
- Second, we could choose \(\mathbf{w}\) to not be on the line \(\text{span}(\mathbf{v})\). In this case, \(\text{span}(\mathbf{w})\) is a different line through the origin. By drawing a picture (or looking at the lecture recording), we can see that if we have two different lines through the origin, then any point in the plane can be written as a vector on the first line, plus a vector on the second line. Therefore, \(\text{span}(\mathbf{v}, \mathbf{w})\) is the entire plane: \(\text{span}(\mathbf{v}, \mathbf{w}) = \mathbb{R}^2\).
- First, we could choose \(\mathbf{w}\) to be on the same line that "contains" \(\mathbf{v}\). In that case, all multiples of \(\mathbf{v}\) and \(\mathbf{w}\) will be on that line, so their sum will also be on the line. In other words, \(\text{span}(\mathbf{v}, \mathbf{w})\) is the same as \(\text{span}(\mathbf{v})\). In a certain sense, \(\mathbf{w}\) is "redundant" in this scenario.
In general, the span of a set of vectors will always be some infinite and "flat" object: a line, a plane, a 3D space, etc. (Except for the span of just the zero vector, in which case all linear combinations are just the zero vector.) However, the "dimension" of that object may not always be the same as the number of vectors, as we saw in the case when \(\mathbf{v}\) and \(\mathbf{w}\) were on the same line. We'll make all this much more precise later on.