Exercise
Consider the (closed) first quadrant in the plane: \[Q = \{ (x, y) \in \mathbb{R}^2 : x \ge 0 \text{ and } y \ge 0 \}.\] Is \(Q\) a subspace of \(\mathbb{R}^2\)? If not, which of the three properties defining a subspace does \(Q\) fail to have?
Today we'll discuss the fundamental concepts of basis and dimension.
Suppose I am thinking of a subspace of \(\mathbb{R}^n.\) How could I tell you what my subspace is? That is, what's a good way for me to represent my subspace?
Well, we know that any subspace can be represented as the span of some list of vectors in \(\mathbb{R}^n.\) So, if I were to give you a list of vectors whose span is equal to my subspace, then you would know what my subspace is. If you then wrote down any vector in \(\mathbb{R}^n\), you would be able to figure out if it's in my subspace by asking whether it lies in the span of the vectors I gave you. (Remember from Chapter 2 that this is equivalent to checking whether a particular linear system is consistent.)
However, not all lists of vectors are equally "good" or "efficient" representations of a subspace. For example, if I am thinking of the \(xy\)-plane inside \(\mathbb{R}^3,\) I could give you the list \[\mathbf{v}_1 = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \quad \mathbf{v}_2 = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, \quad \mathbf{v}_3 = \begin{bmatrix} 0 \\ 2 \\ 0 \end{bmatrix}, \] which is a list of vectors that span the \(xy\)-plane. But my list has some redundancies, because the vectors are linearly dependent: \(\mathbf{v}_3 = 2\mathbf{v}_2.\) This means that I could remove \(\mathbf{v}_3\) from my list and still have the list span the \(xy\)-plane. (Alternatively, I could remove \(\mathbf{v}_2\) from the list because \(\mathbf{v}_2 = \frac{1}{2} \mathbf{v}_3.\)) If I remove \(\mathbf{v}_3,\) then we get the list \[\mathbf{v}_1 = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \quad \mathbf{v}_2 = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}\] which still spans the \(xy\)-plane, but now these vectors are linearly independent. This is, in some sense, a "minimal representation" of this subspace, because every vector is necessary to span the whole subspace.
This idea leads to an extremely important definition:
Definition (basis of a subspace)
Let \(S\) be a subspace of \(\mathbb{R}^n.\) A basis for \(S\) is a list of vectors \(\mathbf{v}_1, \dots, \mathbf{v}_m\) in \(S\) that both are linearly independent and also span \(S.\)
For example, we just saw that the two vectors \(\begin{bmatrix}1\\0\\0 \end{bmatrix}\) and \(\begin{bmatrix}0\\1\\0 \end{bmatrix}\) form a basis for the \(xy\)-plane in \(\mathbb{R}^3.\)
Example (standard basis)
The standard basis vectors \(\mathbf{e}_1, \dots, \mathbf{e}_n \in \mathbb{R}^n\) form a basis for \(\mathbb{R}^n.\)
The plural of "basis" is "bases," pronounced "bay-seez" (spelled like the plural of "base", but not pronounced the same!). This is similar to the plural of "axis" being "axes," or the plural of "hypothesis" being "hypotheses."
In the particular case of \(\mathbb{R}^n,\) we know that for a list of vectors to span \(\mathbb{R}^n\) and be linearly independent, there must be exactly \(n\) of them. Furthermore, if we have \(n\) vectors in \(\mathbb{R}^n,\) then they span \(\mathbb{R}^n\) if and only if they are linearly independent. In other words, we can add to our unifying theorem once again:
The Unifying Theorem, version 4
Let \(A\) be an \(n \times n\) matrix with columns \(\mathbf{v}_1, \dots, \mathbf{v}_n\). Let \(T \colon \mathbb{R}^n \to \mathbb{R}^n\) be the linear transformation defined by \(T(\mathbf{x}) = A\mathbf{x}.\) Then, the following are equivalent:
(i) \(\mathbf{v}_1, \dots, \mathbf{v}_n\) span \(\mathbb{R}^n\).
(ii) \(\mathbf{v}_1, \dots, \mathbf{v}_n\) are linearly independent.
(iii) For all \(\mathbf{b} \in \mathbb{R}^n\), the linear system \(A\mathbf{x} = \mathbf{b}\) has exactly one solution.
(iv) \(T\) is one-to-one.
(v) \(T\) is onto.
(vi) \(A\) is invertible.
(vii) \(\mathbf{v}_1, \dots, \mathbf{v}_n\) form a basis for \(\mathbb{R}^n.\)
The advantage of a basis for a subspace, rather than any spanning set for the subspace, is that we get unique representation of vectors in the subspace:
Fact (unique linear combination of basis vectors)
Suppose that \(S\) is a subspace of \(\mathbb{R}^n\) and that \(\mathbf{v}_1, \dots, \mathbf{v}_m\) is a basis for \(S.\) Then, any vector \(\mathbf{x} \in S\) can be uniquely written as a linear combination of \(\mathbf{v}_1, \dots, \mathbf{v}_m.\)
To see this, first remember that \(\mathbf{x} \in S\) can be written as a linear combination of \(\mathbf{v}_1, \dots, \mathbf{v}_m\) because \(\mathbf{v}_1, \dots, \mathbf{v}_m\) span \(S.\) Second, this linear combination must be unique because \(\mathbf{v}_1, \dots, \mathbf{v}_m\) are linearly independent (this is a fact from Chapter 2).
Because of this fact, a basis of a subspace \(S\) essentially allows us to put a "coordinate system" on \(S\) derived from that basis. For instance, in the plane \(\mathbb{R}^2,\) we know that the standard basis vectors \(\mathbf{e}_1 = (1,0) \) and \(\mathbf{e}_2 = (0, 1)\) form a basis. For any vector \(\mathbf{v} = (x, y)\) in the plane, its coordinates tell us how to write \(\mathbf{v}\) as a linear combination of the standard basis vectors: \[ \mathbf{v} = \begin{bmatrix} x \\ y \end{bmatrix} = x \begin{bmatrix} 1 \\ 0 \end{bmatrix} + y \begin{bmatrix} 0 \\ 1 \end{bmatrix} = x \mathbf{e}_1 + y \mathbf{e}_2. \] But \(\mathbb{R}^2\) has infinitely many other bases! For example, we could take the vectors \[\mathbf{v}_1 = \begin{bmatrix} 1 \\ 1 \end{bmatrix} \quad \text{and} \quad \mathbf{v}_2 = \begin{bmatrix} 0 \\ 2 \end{bmatrix}.\] These are 2 vectors in \(\mathbb{R}^2\), and they are linearly independent, so by the unifying theorem, they span and therefore form a basis for \(\mathbb{R}^2.\) Let's give the basis a name: \(\mathcal{B} = (\mathbf{v}_1, \mathbf{v}_2).\) (That's a calligraphic "B", written in LaTeX as \mathcal{B}.) Because \(\mathcal{B}\) is a basis for \(\mathbb{R}^2,\) any vector in \(\mathbb{R}^2\) can be written uniquely as a linear combination of \(\mathbf{v}_1\) and \(\mathbf{v}_2.\) For example, the vector \(\mathbf{v} = \begin{bmatrix} 5 \\ 1 \end{bmatrix}\) can be written as \[ \mathbf{v} = 5 \mathbf{v}_1 - 2\mathbf{v}_2\] (check this for yourself!). This means that, with respect to the basis \(\mathcal{B},\) the vector \(\mathbf{v}\) has coordinates \((5, -2).\) We can write this as \[\mathbf{v} = \begin{bmatrix} 5 \\ -2 \end{bmatrix}_{\mathcal{B}},\] where we put \(\mathcal{B}\) as a subscript for the vector to indicate that this is meant to be the coordinates with respect to \(\mathcal{B}.\) As another example, \(\mathbf{v}_1\) itself has coordinates \((1,0)\) with respect to the basis, and \(\mathbf{v}_2\) has coordinates \((0,1).\) (This is true for any basis \((\mathbf{v}_1, \mathbf{v}_2).\))
Definition (Coordinates in a basis)
Suppose that \(S\) is a subspace of \(\mathbb{R}^n\) and that \(\mathcal{B} = (\mathbf{v}_1, \dots, \mathbf{v}_m)\) is a basis for \(S.\) Given a vector \(\mathbf{v} \in S,\) the coordinates of \(\mathbf{v}\) with respect to the basis \(\mathbf{v}_1, \dots, \mathbf{v}_m\) are the unique coefficients \(x_1, \dots, x_m\) such that \[\mathbf{v} = x_1 \mathbf{v}_1 + \dotsm + x_m \mathbf{v}_m.\] To signify this, we write \[\mathbf{v} = \begin{bmatrix} x_1 \\ \vdots \\ x_m \end{bmatrix}_{\mathcal{B}}.\]
This may take some time to sink in. (It absolutely did for me!) For now, let's move on, but we'll come back to this topic next week when we discuss a process called change-of-basis.
In general, a subspace will have infinitely different bases. However, the striking thing about them is that they all have the same size (number of vectors).
Theorem (bases have same size)
Suppose that \(S\) is a subspace of \(\mathbb{R}^n\). Then any two bases of \(S\) must contain the same number of vectors.
As an illustration of this, notice that in the case of \(S = \mathbb{R}^n,\) we saw that a basis of \(\mathbb{R}^n\) must contain exactly \(n\) vectors. (If we have fewer than \(n\) vectors, then they can't span \(\mathbb{R}^n\); if we have more than \(n\) vectors, then they can't be linearly independent.) This makes sense, because \(\mathbb{R}^n\) is intuitively "\(n\)-dimensional", so there should be exactly \(n\) "independent directions" in \(\mathbb{R}^n\) that we can travel in.
Because of this theorem (a proof of which you can find in the textbook), it makes sense to define the dimension of a subspace as follows:
Definition (dimension of a subspace)
Let \(S\) be a subspace of \(\mathbb{R}^n\). The dimension of \(S,\) denoted \(\text{dim}(S),\) is the size of any basis of \(S.\)
Because all bases for \(S\) have the same size, it doesn't matter which basis we pick; therefore we say that the dimension is a well-defined quantity.
With this definition, we finally know what we mean when we say, for example, that "the \(xy\)-plane in \(\mathbb{R}^3\) is a two-dimensional subspace": we mean that every basis for the \(xy\)-plane consists of exactly 2 vectors. (I might argue that this is the point of mathematics: taking our valuable human intuition and putting a firm logical foundation under it.)
Examples (dimensions of various subspaces)
- The dimension of \(\mathbb{R}^n\) itself is \(n.\)
- If \(\mathbf{v}\) is a nonzero vector in \(\mathbb{R}^n,\) then the line spanned by \(\mathbf{v}\) is a 1-dimensional subspace of \(\mathbb{R}^n,\) and it has a basis consisting of just \(\mathbf{v}.\)
- The set \(\{\mathbf{0}\}\) is the smallest possible subspace of \(\mathbb{R}^n.\) Its dimension is defined to be zero, and it has a unique basis: the empty set, containing zero vectors. (The textbook instead says that \(\{\mathbf{0}\}\) does not have a basis, however, the textbook is wrong. Luckily this won't matter much for us, as it's a technical point.)
Let's now consider ways to actually compute a basis for a subspace. Assume that we have a subspace of \(\mathbb{R}^n\) given as a span of some list of vectors: \[S = \text{span}(\mathbf{v}_1, \dots, \mathbf{v}_m).\] However, we don't know if the given vectors \(\mathbf{v}_1, \dots, \mathbf{v}_m\) are linearly independent, so they may or may not form a basis for \(S.\) How can we take these vectors and use them to find a basis for \(S\)? Here are two methods to do this.
The idea of this method is as follows:
As an example, suppose we are given four vectors in \(\mathbb{R}^3\): \[\mathbf{v}_1 = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}, \quad \mathbf{v}_2 = \begin{bmatrix} 2 \\ 0 \\ 2 \end{bmatrix}, \quad \mathbf{v}_3 = \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}, \quad \mathbf{v}_4 = \begin{bmatrix} -1 \\ -1 \\ 1 \end{bmatrix}.\] (In this case, we in fact know that these can't be linearly independent, because they are four vectors in \(\mathbb{R}^3.\)) Then we make the \(3 \times 4\) matrix \[A = \begin{bmatrix} 1 & 2 & 2 & -1 \\ 0 & 0 & 1 & -1 \\ 1 & 2 & 0 & 1 \end{bmatrix}\] and use row reduction to get to echelon form: \[\begin{bmatrix} 1 & 2 & 2 & -1 \\ 0 & 0 & 1 & -1 \\ 1 & 2 & 0 & 1 \end{bmatrix} \stackrel{-R_1+R_3 \to R_3}\longrightarrow \begin{bmatrix} 1 & 2 & 2 & 1 \\ 0 & 0 & 1 & -1 \\ 0 & 0 & -2 & 2 \end{bmatrix} \stackrel{2R_2+R_3 \to R_3}\longrightarrow \begin{bmatrix} 1 & 2 & 2 & 1 \\ 0 & 0 & 1 & -1 \\ 0 & 0 & 0 & 0 \end{bmatrix} = B.\] This (reduced) echelon-form matrix has pivots in columns 1 and 3, so this means that \(\mathbf{v}_1, \mathbf{v}_3\) is a basis for \(\text{span}(\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3, \mathbf{v}_4).\) (Even though we started with four vectors in \(\mathbb{R}^3,\) we end up only getting a two-dimensional subspace of \(\mathbb{R}^3.\))
Why does this work? The key idea is that row operations on a matrix preserve linear relations between the columns. For instance, in \(B\), the second column equals twice the first column. Then the same should be true with the original matrix \(A\), and indeed, we see that \(\mathbf{v}_2 = 2\mathbf{v}_1.\) As another example, the fourth column \(B\) equals the first column minus the third column; indeed, the same is true in \(A\), as \(\mathbf{v}_4 = \mathbf{v}_1 - \mathbf{v}_3.\) The reason for this is simple: linear relations between the columns correspond to solutions to \(A \mathbf{x} = \mathbf{0},\) which row operations are designed to preserve. Indeed, if \(c_1 \mathbf{v}_1 + \dotsm + c_m \mathbf{v}_m = 0,\) then \[\begin{bmatrix} | & & | \\ \mathbf{v}_1 & \dots & \mathbf{v}_m \\ | & & | \end{bmatrix} \begin{bmatrix} c_1 \\ \vdots \\ c_m \end{bmatrix} = \mathbf{0},\] and vice versa.
Now, because \(B\) is in echelon form, its pivot columns must be linearly independent. (One way to see this is, if you take the matrix consisting of only the pivot columns, then that matrix will obviously have a pivot in every column!) Furthermore, all the non-pivot columns are in the span of the pivot columns. Therefore the same is true for the columns of \(A\): namely, in our case, \(\mathbf{v}_1\) and \(\mathbf{v}_3\) are linearly independent, and \(\mathbf{v}_2, \mathbf{v}_4\) lie in \(\text{span}(\mathbf{v}_1, \mathbf{v}_3).\) This means that \(\mathbf{v}_1\) and \(\mathbf{v}_3\) form a linearly independent and spanning set for \(\text{span}(\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3, \mathbf{v}_4),\) that is, a basis.
The second method works as follows:
Let's use this method on the same example: \[\mathbf{v}_1 = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}, \quad \mathbf{v}_2 = \begin{bmatrix} 2 \\ 0 \\ 2 \end{bmatrix}, \quad \mathbf{v}_3 = \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}, \quad \mathbf{v}_4 = \begin{bmatrix} -1 \\ -1 \\ 1 \end{bmatrix}.\] In this case, we are forming a \(4 \times 3\) matrix \[A = \begin{bmatrix} 1 & 0 & 1 \\ 2 & 0 & 2 \\ 2 & 1 & 0 \\ -1 & -1 & 1 \end{bmatrix},\] which is the same as the matrix from the first method but with the rows and columns switched. (This is called the transpose of the above matrix.) Now let's transform \(A\) to echelon form: \[\begin{bmatrix} 1 & 0 & 1 \\ 2 & 0 & 2 \\ 2 & 1 & 0 \\ -1 & -1 & 1 \end{bmatrix} \stackrel{-2R_1+R_2 \to R_2}\longrightarrow \begin{bmatrix} 1 & 0 & 1 \\ 0 & 0 & 0 \\ 2 & 1 & 0 \\ -1 & -1 & 1 \end{bmatrix} \stackrel{-2R_1+R_3 \to R_3}\longrightarrow \begin{bmatrix} 1 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 1 & -2 \\ -1 & -1 & 1 \end{bmatrix} \stackrel{R_1+R_4 \to R_4}\longrightarrow \begin{bmatrix} 1 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 1 & -2 \\ 0 & -1 & 2 \end{bmatrix} \stackrel{R_2 \leftrightarrow R_4} \longrightarrow \begin{bmatrix} 1 & 0 & 1 \\ 0 & -1 & 2 \\ 0 & 1 & -2 \\ 0 & 0 & 0 \end{bmatrix} \longrightarrow \stackrel{R_2+R_3\to R_3}\longrightarrow \begin{bmatrix} 1 & 0 & 1 \\ 0 & -1 & 2 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}.\] This means that \(\begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}\) and \(\begin{bmatrix} 0 \\ -1 \\ 2 \end{bmatrix}\) form a basis for \(\text{span}(\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3, \mathbf{v}_4).\)
Why does this work? This one is simpler to understand: all three types of row operations preserve the span of the rows (i.e., the row space of the matrix). Furthermore, in echelon form, all the nonzero rows are linearly independent. (Indeed, if we keep only the nonzero rows, then every row will have a pivot by the definition of "pivot." This means that the rows are linearly independent, in the same way that if a matrix has a pivot in every column then the columns are linearly independent.) So, we can simply take the nonzero rows, which are linearly independent and have the same span as the original vectors \(\mathbf{v}_1, \dots, \mathbf{v}_m,\) so they form a basis for \(\text{span}(\mathbf{v}_1, \dots, \mathbf{v}_m).\)
Both of these methods work for extracting a basis of a subspace from a spanning set. However, as in the example above, they generally will not produce the same basis. Each has a particular advantage that can be useful depending on the situation: