Pre-Class Warmup
Suppose that \(A\) is a \(5 \times 4\) matrix such that \[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 2 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}.\] Which of the following can we find a basis for: \(\text{col}(A),\) \(\text{row}(A),\) and \(\text{null}(A)\)?
Today we will take another look at the three subspaces associated to a matrix, and consider the notions of rank and nullity.
Last week, we looked at methods to find a basis for the column space and the row space of a matrix \(A.\) Both methods involved putting \(A\) into echelon form with row operations, at which point we could use the pivots to read off a basis:
These methods prove a general theorem relating the column space and row space:
Theorem (dimensions of column space and row space)
For any matrix \(A,\) the dimensions of \(\text{col}(A)\) and \(\text{row}(A)\) are always the same, and are equal to the number of pivots of \(A\) when put in echelon form.
(This is a remarkable fact! Notice that \(\text{col}(A)\) and \(\text{row}(A)\) do not even live in the same space: if \(A\) is \(n \times m\), then \(\text{col}(A)\) is a subspace of \(\mathbb{R}^n,\) while \(\text{row}(A)\) is a subspace of \(\mathbb{R}^m.\) However, they still have the same dimension as subspaces of their corresponding Euclidean spaces.)
Since \(\text{dim col}(A)\) and \(\text{dim row}(A)\) are the same number, we can give that number a name:
Definition (rank)
The rank of a matrix \(A\), denoted \(\text{rank}(A),\) is the dimension of its column space and row space. Equivalently, it is the number of pivots of \(A\) when put in echelon form.
If \(A\) is an \(n \times m\) matrix, then \(\text{rank}(A)\) is a nonnegative integer. Because it equals the dimension of two different subspaces, one a subspace of \(\mathbb{R}^n\) and the other a subspace of \(\mathbb{R}^m,\) it must be at most \(n\) and at most \(m.\) In other words, we must have \[0 \le \text{rank}(A) \le \text{min}(m, n).\] (Another way to see this: \(A\) can only have at most one pivot in each of its \(n\) rows, and at most one pivot in each of its \(m\) columns, so it can have at most \(\text{min}(m,n)\) pivots total.)
We also use the word rank for the linear transformation \(T(\mathbf{x}) = A\mathbf{x}\) corresponding to a matrix \(A.\) (That is, the rank of a linear transformation is defined to be the rank of its matrix.) Remember that the column space of \(A\) is the same as the range of \(T.\) Therefore, the rank of a linear transformation is \[\text{rank}(T) = \text{dim range}(T).\] Intuitively, the rank of a matrix quantifies "how much" of the codomain gets hit by the corresponding linear transformation. The higher the rank, the "larger" the range of the linear transformation.
In fact, if \(A\) is \(n \times m\) and \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) is the corresponding linear transformation, then we have the following equivalences: \[T \text{ is onto} \quad\Leftrightarrow\quad \text{range}(T) = \mathbb{R}^n \quad\Leftrightarrow\quad \text{dim range}(T) = n \quad\Leftrightarrow\quad \text{rank}(T) = n.\] (Notice that if \(m < n,\) then we know that \(\text{rank}(T) \le \text{min}(m,n) = m,\) so in particular the rank of \(T\) must be strictly less than \(n.\) Therefore, \(T\) cannot be onto when \(m < n,\) which is something we already knew.)
On the other extreme, if \(\text{rank}(T) = 0,\) then the range of \(T\) must be the subspace of \(\mathbb{R}^n\) consisting only of the zero vector. In other words, \(T\) must send every vector in its domain to \(\mathbf{0}.\) (In this case, the corresponding matrix \(A\) has all zeros, and so there are no pivots.)
We also have a term for the dimension of the null space of a matrix:
Definition (nullity)
The nullity of a matrix is the dimension of its null space: \[\text{nullity}(A) = \text{dim null}(A).\]
Recall that the null space of an \(n \times m\) matrix is a subspace of \(\mathbb{R}^m.\) Therefore, we have \[0 \le \text{nullity}(A) \le m.\] Another way to think of the nullity comes from the method we used to find the null space of a matrix (see last class). Specifically, when computing \(\text{null}(A),\) we solve the system \(A\mathbf{x} = \mathbf{0}\) and write the resulting solution set as the span of linearly independent vectors. Because each of those vectors corresponds to a free variable in the system, we have the following fact:
Fact (nullity = number of free variables)
The nullity of a matrix \(A\) equals the number of free variables in the system \(A\mathbf{x} = \mathbf{0}\) (when put in echelon form). Equivalently, it equals the number of non-pivot columns in \(\text{rref}(A)\).
Like rank, we also use the word nullity for the linear transformation associated to a matrix. The null space of a matrix corresponds to the kernel of its linear transformation, so \[\text{nullity}(T) = \text{dim ker}(T).\] (Remember that the kernel of \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) is the subspace of all vectors \(\mathbf{x} \in \mathbb{R}^m\) such that \(T(\mathbf{x}) = 0.\))
Intuitively, the nullity of a matrix tells us how many dimensions get sent to zero by the corresponding linear transformation. The higher the nullity, the more things get sent to zero. In particular, recall that a linear transformation \(T\) is one-to-one if and only if the only solution to \(T(\mathbf{x}) = \mathbf{0}\) is \(\mathbf{x} = \mathbf{0}\) (the only vector that gets sent to zero is the zero vector). In other words, \(T\) is one-to-one if and only if \(\text{ker}(T) = \{ \mathbf{0} \}.\) Therefore, we have the following chain of equivalences: \[T \text{ is one-to-one} \quad\Leftrightarrow\quad \text{ker}(T) = \{\mathbf{0}\} \quad\Leftrightarrow\quad \text{dim ker}(T) = 0 \quad\Leftrightarrow\quad \text{nullity}(T) = 0.\] On the other extreme, if \(\text{nullity}(T) = m,\) then \(\text{ker}(T)\) must be all of \(\mathbb{R}^m,\) which means that \(T\) sends every vector in its domain to \(\mathbf{0}.\)
Example (rank and nullity)
Consider the \(5 \times 4\) matrix \(A\) from the pre-class warmup. We don't know \(A\), but we are given that \[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 2 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}.\] Using this, we see that \(\text{rref}(A)\) has three pivots, so \(\text{rank}(A) = 3.\) On the other hand, \(\text{rref}(A)\) has one column without a pivot, so \(\text{nullity}(A) = 1.\)
Based on all this discussion, we can update our Unifying Theorem for \(n \times n\) matrices and linear transformations \(\mathbb{R}^n \to \mathbb{R}^n\):
The Unifying Theorem, version 5
Let \(A\) be an \(n \times n\) matrix with columns \(\mathbf{v}_1, \dots, \mathbf{v}_n\). Let \(T \colon \mathbb{R}^n \to \mathbb{R}^n\) be the linear transformation defined by \(T(\mathbf{x}) = A\mathbf{x}.\) Then, the following are equivalent:
(i) \(\mathbf{v}_1, \dots, \mathbf{v}_n\) span \(\mathbb{R}^n\).
(ii) \(\mathbf{v}_1, \dots, \mathbf{v}_n\) are linearly independent.
(iii) For all \(\mathbf{b} \in \mathbb{R}^n\), the linear system \(A\mathbf{x} = \mathbf{b}\) has exactly one solution.
(iv) \(T\) is one-to-one.
(v) \(T\) is onto.
(vi) \(A\) is invertible.
(vii) \(\mathbf{v}_1, \dots, \mathbf{v}_n\) form a basis for \(\mathbb{R}^n.\)
(viii) \(\text{col}(A) = \text{range}(T) = \mathbb{R}^n.\)
(ix) \(\text{row}(A) = \mathbb{R}^n.\)
(x) \(\text{rank}(A) = \text{rank}(T) = n.\)
(xi) \(\text{null}(A) = \text{ker}(A) = \{\mathbf{0}\}.\)
(xii) \(\text{nullity}(A) = \text{nullity}(T) = 0.\)
Finally, let's take a look at a fundamental result in linear algebra that relates the rank and nullity of a matrix, the rank-nullity theorem.
Theorem (rank-nullity)
Let \(A\) be an \(n \times m\) matrix. Then \[\text{rank}(A) + \text{nullity}(A) = m.\]
To prove this, we only need to combine two things that we've learned:
Therefore, the sum of the rank and nullity must equal the number of columns in \(A,\) which is \(m.\) Done!
To understand the rank-nullity theorem geometrically / intuitively, it helps to think about the linear transformation \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) corresponding to \(A.\) In terms of \(T,\) the rank-nullity theorem says that \(\text{rank}(T) + \text{nullity}(T) = m,\) or \[\text{dim range}(T) + \text{dim ker}(T) = m.\] The key idea is "conservation of dimension." The domain of \(T\) is \(\mathbb{R}^m,\) an \(m\)-dimensional space, and \(T\) maps this space to some subspace of \(\mathbb{R}^n\), which is its range. The rank of \(T\) measures how many dimensions are in the range, while the nullity of \(T\) measures how many dimensions were "lost" (i.e., got sent to zero) when applying the linear transformation. Together, these must sum to the number of dimensions we started with, which is the dimension of the domain \(\mathbb{R}^m.\)
Example (rank-nullity 1)
Consider again the \(5 \times 4\) matrix \(A\) with reduced echelon form \[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 2 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}.\] We saw above that \(A\) has rank \(3\) and nullity \(1.\) This corresponds to a linear transformation \(T \colon \mathbb{R}^4 \to \mathbb{R}^5\) whose range is 3-dimensional and whose kernel is 1-dimensional. And indeed, \(3 + 1 = 4\) matches the dimension of the domain of \(T.\)
Intuitively, when applying \(T\), we start with a 4-dimensional space, and this computation shows that we end up with a 3-dimensional range. Then there was "one dimension lost," which corresponds to the fact that the kernel of \(T\) is 1-dimensional.
Here are more examples of how the rank-nullity theorem can be used to draw conclusions about a matrix or its linear transformation.
Example (rank-nullity 2)
Suppose that \(T \colon \mathbb{R}^6 \to \mathbb{R}^4\) is a linear transformation and that \(T\) is onto. What can we say about the kernel of \(T\)?
Because \(T\) is onto, the range of \(T\) is all of \(\mathbb{R}^4,\) which means that \(\text{rank}(T) = 4.\) Now, by the rank-nullity theorem, we have \[\text{rank}(T) + \text{nullity}(T) = 6,\] so we must have \[\text{nullity}(T) = 6 - 4 = 2.\] In other words, the kernel of \(T\) must be 2-dimensional!
Example (rank-nullity 3)
Let \(T \colon \mathbb{R}^3 \to \mathbb{R}^2\) be a linear transformation. Suppose that \[T\left(\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}\right) = T\left(\begin{bmatrix} 4 \\ 0 \\ 1 \end{bmatrix}\right) = \begin{bmatrix} 0 \\ 0 \end{bmatrix}\] and that the vector \(\begin{bmatrix} 1 \\ 1 \end{bmatrix}\) is in the range of \(T.\) Can we determine the rank and nullity of \(T\)?
What we are given is that \(\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}\) and \(\begin{bmatrix} 4 \\ 0 \\ 1 \end{bmatrix}\) lie in the kernel of \(T.\) Since these vectors are linearly independent (not multiples of each other), this means that the kernel of \(T\) is at least 2-dimensional, or in other words, the nullity of \(T\) is at least 2. On the other hand, because we are given a nonzero vector in the range of \(T,\) we know that the rank of \(T\) must be at least 1.
But we know that \[ \text{rank}(T) + \text{nullity}(T) = 3 \] by the rank-nullity theorem! Therefore, the only possibility is that the rank of \(T\) is exactly 1 and the nullity of \(T\) is exactly 2. (Intuitively, by finding "two dimensions' worth" of vectors that \(T\) sends to zero, and "one dimension's worth" of vectors that gets mapped onto by \(T,\) we've "used up" all three dimensions in the domain of \(T.\))
As a consequence of this, we know that \[\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} \quad \text{and} \quad \begin{bmatrix} 4 \\ 0 \\ 1 \end{bmatrix}\] form a basis for \(\text{ker}(T),\) because they are two linearly independent vectors and \(\text{ker}(T)\) is 2-dimensional. Similarly, \(\begin{bmatrix} 1 \\ 1 \end{bmatrix}\) must be a basis for \(\text{range}(T),\) since the range is 1-dimensional.