This week and next we'll cover Chapter 4 of the book, which examines certain subspaces associated to a matrix and the concepts of basis and dimension.

Subspaces of Euclidean space

Let's begin by considering subspaces of \(\mathbb{R}^n.\) These are subsets of \(\mathbb{R}^n\) with certain nice properties, that geometrically look like lines / planes / higher-dimensional hyperplanes passing through the origin.

Definition (subspace of \(\mathbb{R}^n\))

A subspace of \(\mathbb{R}^n\) is a subset \(S \subseteq \mathbb{R}^n\) that satisfies all of the following three properties:

(i) \(S\) contains the zero vector: \(\mathbf{0} \in S\).

(ii) \(S\) is closed under addition: if two vectors \(\mathbf{u}, \mathbf{v}\) are in \(S\), then their sum \(\mathbf{u} + \mathbf{v}\) must also be in \(S\).

(iii) \(S\) is closed under scalar multiplication: if a vector \(\mathbf{u}\) is in \(S\), then \(c \mathbf{u}\) must be in \(S\) for all scalars \(c\).

Notice that the last two properties are very similar to the key properties of linear transformations (namely, a linear transformation must respect addition and scalar multiplication). However, here, we are talking about a set of vectors, not a function whose input and output are vectors.

Example ("obvious" subspaces)

(1) The set \( \{ \mathbf{0} \} \), containing only the zero vector, is always a subspace of \(\mathbb{R}^n.\) Clearly this set satisfies property (i), and it is closed under addition because the only possible sum of two vectors from the set is \(\mathbf{0} + \mathbf{0} = \mathbf{0}.\) It is closed under scalar multiplication because any scalar multiple of \(\mathbf{0}\) is still \(\mathbf{0}.\)

(2) The set \(\mathbb{R}^n\) itself (containing all vectors with \(n\) components) is always a subspace of \(\mathbb{R}^n,\) just by the definition of addition and scalar multiplication (they always produce vectors in \(\mathbb{R}^n\)).

Subspaces and spans

We can imagine trying to construct a subspace \(S\) of \(\mathbb{R}^n\) by adding vectors one-by-one. To get a sense of what subspaces look like, let's consider how this might go:

• To start, \(S\) must contain the zero vector \(\mathbf{0}\) by property (i). If we wanted to, we could stop here, as we just saw that \(\{ \mathbf{0} \} \) is a valid subspace.
  
  
• But maybe we want a more interesting subspace, so let's take some nonzero vector \(\mathbf{v} \in \mathbb{R}^n\) and add it to \(S.\) Now \[S = \{\mathbf{0}, \mathbf{v}\}.\] Is this a valid subspace of \(\mathbb{R}^n\)? No! It doesn't satisfy either of properties (ii) or (iii). For example, property (ii) says that \(S\) must also contain \(\mathbf{v} + \mathbf{v} = 2\mathbf{v}.\) Even more generally, property (iii) says that \(S\) must contain all scalar multiples of \(\mathbf{v}.\) If we add all scalar multiples of \(\mathbf{v}\) to \(S\), then we get \[ S = \{ c\mathbf{v} : c \in \mathbb{R} \} = \text{span}(\mathbf{v}). \] (Note that \(\mathbf{0}\) is a scalar multiple of \(\mathbf{v}\), so we don't need to include it separately.) Is this a subspace of \(\mathbb{R}^n\)? Yes! The sum of two multiples of \(\mathbf{v}\) is still a multiple of \(\mathbf{v},\) and any multiple of a multiple of \(\mathbf{v}\) is still a multiple of \(\mathbf{v}.\) Geometrically, \(S\) is the line through the origin spanned by \(\mathbf{v}.\) So we could stop here.
  
  
• But maybe we want to go even further. What if we add another vector \(\mathbf{w}\) to \(S\), where \(\mathbf{w}\) is not in \(\text{span}(\mathbf{v})\)? (Notice that since \(\mathbf{v}\) is nonzero and \(\mathbf{w}\) is not in \(\text{span}(\mathbf{v}),\) this means that \(\mathbf{v}, \mathbf{w}\) are linearly independent.)
  
  So now we have \(S = \text{span}(\mathbf{v}) \cup \{ \mathbf{w} \}.\) Is this a subspace? No! Once again, \(S\) is no longer closed under addition or scalar multiplication. From property (iii), we know that \(S\) should contain all scalar multiples of \(\mathbf{w}.\) Adding those to our set, we now have \(S = \text{span}(\mathbf{v}) \cup \text{span}(\mathbf{w}).\)  However, that isn't enough, because now property (ii) says that \(S\) should contain every sum of a vector in \(\text{span}(\mathbf{v}) \) and \(\text{span}(\mathbf{w}) .\) In other words, \(S\) should contain every linear combination of \(\mathbf{v}\) and \(\mathbf{w}\)! So now we are forced to take \[ S = \text{span}(\mathbf{v}, \mathbf{w}).\] Is this a subspace? Yes! If we take any two linear combinations of \(\mathbf{v}\) and \(\mathbf{w}\) and add them, then the result is still a linear combination of \(\mathbf{v}\) and \(\mathbf{w}\). Similarly, any scalar multiple of an element of \(\text{span}(\mathbf{v}, \mathbf{w})\) is still in \(\text{span}(\mathbf{v}, \mathbf{w}).\) So we could stop here, and this would give us the plane through the origin spanned by \(\mathbf{v}\) and \(\mathbf{w}.\)
  
  
• We could keep playing this game, repeatedly adding a new vector to our set and seeing what we get. As you might guess, each time we do this, we'll end up having to include the entire span of all the vectors we chose. Eventually, we will have to stop: the vectors we choose to add will all be linearly independent, since we make sure that each one is not in the span of the previous ones. So by the unifying theorem, once we choose \(n\) linearly independent vectors, their span will be all of \(\mathbb{R}^n,\) and we won't have any more vectors to add.

This process suggests that subspaces of \(\mathbb{R}^n\) are closely related to spans of vectors in \(\mathbb{R}^n.\) Indeed, we have the following theorem:

Theorem (subspaces and spans)

For any vectors \(\mathbf{v}_1, \dots, \mathbf{v}_m\) in \(\mathbb{R}^n\), their span \(\text{span}(\mathbf{v}_1, \dots, \mathbf{v}_m)\) is a subspace of \(\mathbb{R}^n.\)

Conversely: if \(S\) is any subspace of \(\mathbb{R}^n,\) then there exist vectors \(\mathbf{v}_1, \dots, \mathbf{v}_m\) such that \(S = \text{span}(\mathbf{v}_1, \dots, \mathbf{v}_m).\)

Subspaces and homogeneous systems

Another way that subspaces arise is as solution sets of homogeneous systems:

Theorem (subspaces and homogeneous systems)

Let \(A\) be an \(n \times m\) matrix, and consider the homogeneous linear system \(A\mathbf{x} = \mathbf{0}.\) Then the set of all solutions for \(\mathbf{x}\) forms a subspace of \(\mathbb{R}^m.\)

To prove this, we need to show that the solution set satisfies the three properties of a subspace:

• [Contains zero vector] Because \(A \mathbf{0} = \mathbf{0},\) the solution set contains the zero vector.
• [Closure under addition] Suppose that we have two solutions to the system, say \(\mathbf{x}\) and \(\mathbf{x}'.\) Then we have \(A\mathbf{x} = \mathbf{0}\) and \(A\mathbf{x}' = \mathbf{0}.\) By adding these two equations and using the distributive property, we get \[A(\mathbf{x} + \mathbf{x}') = \mathbf{0},\] which means that \(\mathbf{x} + \mathbf{x'}\) is also a solution, so it is in the solution set.
• [Closure under scalar multiplication] Suppose we have a solution \(\mathbf{x}\) and a scalar \(c \in \mathbb{R}.\) Then, \(A\mathbf{x} = \mathbf{0}\). Multiplying by \(c\), we get \(c \cdot A\mathbf{x} = \mathbf{0}\), which we can rewrite using properties of matrix multiplication as \(A(c \mathbf{x}) = \mathbf{0}.\) Therefore, \(c\mathbf{x}\) is also a solution to the system, so it is in the solution set.

This shows that the solution set to \(A\mathbf{x} = \mathbf{0}\) is a subspace of \(\mathbb{R}^m,\) as we wanted.

Note: The converse statement of this theorem is also true: any subspace of \(\mathbb{R}^m\) can be expressed as the solution set of a homogeneous system in \(m\) variables. However, we won't have time to get into the proof.

Example (subspace in two ways)

Let \(P\) be the \(xy\)-plane in \(\mathbb{R}^3\) (with coordinates \(x, y, z\)). Because it is a plane passing through the origin, \(P\) should be a subspace of \(\mathbb{R}^3.\) To see why, we can try to write it either as the span of some vectors, or as the solution set of a homogeneous system. (Both of these will always be possible if we have a subspace of \(\mathbb{R}^m.\))

• First, \(P\) is the span of the two vectors \((1,0,0)\) and \((0,1,0)\), so it is a subspace of \(\mathbb{R}^3.\)
• Second, \(P\) is the solution set to the homogeneous system with only one equation, \(z = 0,\) so it is a subspace of \(\mathbb{R}^3.\) In other words, \(P\) is the null space of the matrix \(\begin{bmatrix} 0 & 0 & 1 \end{bmatrix}.\)

What about non-homogeneous systems?

Suppose we instead have a non-homogeneous (or inhomogeneous) system: \[A\mathbf{x} = \mathbf{b},\] where \(A\) is an \(n \times m\) matrix and \(\mathbf{b} \neq \mathbf{0}.\) The solution set to this system is a subset of \(\mathbb{R}^m,\) but it is not a subspace. The easiest way to see this is to notice that the zero vector is not a solution to the system, because \[A \mathbf{0} = \mathbf{0} \neq \mathbf{b}.\] However, if the system is consistent, its solution set will still look "flat": it'll be a point, or line, or plane, etc., but it won't pass through the origin. (Such sets are sometimes called affine subspaces of \(\mathbb{R}^m\), as they aren't regular subspaces.)

Example (non-homogeneous system)

Consider the non-homogeneous system \[ \begin{bmatrix} 1 & 2 & 0 \\ 0 & 0 & 1 \end{bmatrix} \mathbf{x} = \begin{bmatrix} 4 \\ 7 \end{bmatrix}.\] (I chose one in reduced echelon form so we don't need to do any row reduction.) This has \(x_2\) as a free variable, so we can see that the solution set to this equation is given by \[\mathbf{x} = \begin{bmatrix} 4 - 2t \\ t \\ 7\end{bmatrix}.\] Rewriting this to separate out the parts with \(t\), we get \[\mathbf{x} = \begin{bmatrix} 4 \\ 0 \\ 7 \end{bmatrix} + t \begin{bmatrix} -2 \\ 1 \\ 0 \end{bmatrix}\] where \(t\) can be any real number. This is the line in \(\mathbb{R}^3\) spanned by \((-2,1,0)\), but translated by the vector \((4,0,7),\) so it no longer passes through the origin.

In general, suppose we have a consistent non-homogeneous system \(A\mathbf{x} = \mathbf{b},\) and we choose any one particular solution to the system, say \(\mathbf{x}_p\) ("p" for "particular"). Then, we can write any (other) solution \(\mathbf{x}\) as a sum \[\mathbf{x} = \mathbf{x}_p + (\mathbf{x} - \mathbf{x}_p).\] Now, notice that \[A(\mathbf{x} - \mathbf{x}_p) = A\mathbf{x} - A\mathbf{x}_p = \mathbf{b} - \mathbf{b} = \mathbf{0},\] because both \(\mathbf{x}\) and \(\mathbf{x}_p\) are solutions to the given system. In other words, the difference \(\mathbf{x} - \mathbf{x}_p\) is a solution to the associated homogeneous system \(A\mathbf{x} = \mathbf{0}\): the system with the same matrix but where the right-hand side is all zeros. Letting \(\mathbf{x} - \mathbf{x}_p = \mathbf{x}_h\) ("h" for "homogeneous"), we get the following theorem:

Theorem (particular and homogeneous solutions)

Let \(A\mathbf{x} = \mathbf{b}\) be a non-homogeneous system, and suppose \(\mathbf{x}_p\) is a solution. Then, the solutions to the system are given by \[\mathbf{x} = \mathbf{x}_p + \mathbf{x}_h,\] where \(\mathbf{x}_h\) is a solution to the associated homogeneous system \(A\mathbf{x} = \mathbf{0}.\)

Because the solution set of \(A\mathbf{x} = \mathbf{0}\) is a subspace, this shows that the solution set of a non-homogeneous system is always the translation of a subspace.

Subspaces from matrices and linear transformations

Finally, let's define some subspaces that arise naturally from matrices and their corresponding linear transformations. We'll have a lot more to say about these in the next few days.

Definition (column space, row space, null space)

Let \(A\) be an \(n \times m\) matrix.

The column space of \(A\), denoted \(\text{col}(A),\) is the span of the columns of \(A\), considered as vectors in \(\mathbb{R}^n.\)

The row space of \(A\), denoted \(\text{row}(A),\) is the span of the rows of \(A\), considered as vectors in \(\mathbb{R}^m.\)

The null space of \(A\), denoted \(\text{null}(A),\) is the set of all vectors \(\mathbf{x} \in \mathbb{R}^m\) such that \(A\mathbf{x} = \mathbf{0}.\) 

From what we've learned, we now know that all of these are subspaces (of \(\mathbb{R}^n,\) \(\mathbb{R}^m,\) and \(\mathbb{R}^m,\) respectively).

Definition (range, kernel)

Let \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) be a linear transformation.

The range of \(T\), denoted \(\text{range}(T),\) is the set of all vectors in \(\mathbb{R}^n\) of the form \(T(\mathbf{x})\) for \(\mathbf{x} \in \mathbb{R}^m.\)

The kernel of \(T\), denoted \(\text{ker}(T),\) is the set of all vectors \(\mathbf{x} \in \mathbb{R}^m\) such that \(T(\mathbf{x}) = \mathbf{0}.\)

Notice that, if \(A\) is the matrix of \(T,\) then we have \[ \text{range}(T) = \text{col}(A)\] and \[ \text{ker}(T) = \text{null}(A).\] So, the range and kernel of a linear transformation \(\mathbb{R}^m \to \mathbb{R}^n\) are also subspaces (of \(\mathbb{R}^n\) and \(\mathbb{R}^m\), respectively).