Today we will continue discussing linear transformations, which we started to talk about last Wednesday before the midterm.
Here is a quick review of what we went over last time:
A linear transformation is a function \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) satisfying two key conditions:
There is an equivalent way to define linear transformation, namely, it is a function \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) that respects linear combinations: \[T(c\mathbf{u} + d\mathbf{v}) = c\cdot T(\mathbf{u}) + d\cdot T(\mathbf{v})\] for all vectors \(\mathbf{u}, \mathbf{v} \in \mathbb{R}^m\) and all scalars \(c, d \in \mathbb{R}\).
We also saw that every \(n \times m\) matrix \(A\) corresponds to a linear transformation \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) where \(T\) is defined by \[T(\mathbf{v}) = A \mathbf{v}.\] (To be fancy, we can say that an \(n \times m\) matrix induces a linear transformation from \(\mathbb{R}^m\) to \(\mathbb{R}^n\).) For instance, the \(3 \times 2\) matrix \(A = \begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix}\) corresponds to the linear transformation \(T \colon \mathbb{R}^2 \to \mathbb{R}^3\) given by \[T\left(\begin{bmatrix} x \\ y \end{bmatrix}\right) = \begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = x \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} + y \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix} = \begin{bmatrix} x + 4y \\ 2x + 5y \\ 3x + 6y \end{bmatrix}.\] The last thing we did was ask the "reverse" question. Namely, if I have any linear transformation \(T \colon \mathbb{R}^m \to \mathbb{R}^n\), does there have to exist an \(n \times m\) matrix \(A\) that corresponds to it? Or are there some "weird" linear transformations that can't be represented by matrices?
Well, the answer, as it turns out, is...
In order to see why, we will need an additional definition.
Definition
In \(\mathbb{R}^m\), the standard basis vectors are the following \(m\) vectors: \[\begin{aligned} \mathbf{e}_1 &= (1, 0, 0, \dots, 0), \\ \mathbf{e}_2 &= (0, 1, 0, \dots, 0), \\ \mathbf{e}_3 &= (0, 0, 1, \dots, 0), \\ &\vdots \\ \mathbf{e}_m &= (0, 0, 0, \dots, 1). \end{aligned}\] That is, \(\mathbf{e}_i\) is the vector with \(i\)th component \(1\) and all other components \(0.\)
The word "basis" here has a special meaning: these vectors are linearly independent and span \(\mathbb{R}^m\). Therefore, any \(\mathbf{x} \in \mathbb{R}^m\) can be uniquely written as a linear combination of \(\mathbf{e}_1, \dots, \mathbf{e}_m\). (Recall from our discussion last week that this can only happen if we have exactly \(m\) vectors. We will see the word basis much more later on in the course.)
In fact, it's easy to see how to write any vector in \(\mathbb{R}^m\) as a linear combination of \(\mathbf{e}_1, \dots, \mathbf{e}_m\): \[\begin{aligned} \mathbf{x} &= (x_1, x_2, \dots, x_m) \\ &= (x_1, 0, \dots, 0) + (0, x_2, \dots, 0) + \dotsm + (0, 0, \dots, x_m) \\ &= x_1 \mathbf{e}_1 + x_2 \mathbf{e}_2 + \dotsm + x_m \mathbf{e}_m. \end{aligned}\] So, the linear combination is just the one where the components of \(\mathbf{x}\) are the coefficients.
Now, suppose we have any linear transformation \(T \colon \mathbb{R}^m \to \mathbb{R}^n.\) Because \(T\) respects addition and scalar multiplication, we can apply \(T\) to \(\mathbf{x}\) and simplify in the following way: \[\begin{aligned} T(\mathbf{x}) &= T(x_1 \mathbf{e}_1 + x_2 \mathbf{e}_2 + \dotsm + x_m \mathbf{e}_m) \\ &= T(x_1 \mathbf{e}_1) + T(x_2\mathbf{e}_2) + \dotsm + T(x_m\mathbf{e}_m) \\ &= x_1 T(\mathbf{e}_1) + x_2 T(\mathbf{e}_2) + \dotsm + x_m T(\mathbf{e}_m). \end{aligned}\] That is, \(T(\mathbf{x})\) must equal the linear combination of \(T(\mathbf{e}_1), T(\mathbf{e}_2), \dots, T(\mathbf{e}_m)\) whose coefficients are equal to the components of \(\mathbf{x}.\) But this is exactly how we defined a matrix times \(\mathbf{x}\): \[T(\mathbf{x}) = \begin{bmatrix} | & | & & | \\ T(\mathbf{e}_1) & T(\mathbf{e}_2) & \dotsm & T(\mathbf{e}_m) \\ | & | & & | \end{bmatrix} \mathbf{x}.\] This shows that a linear transformation \(T\) is always given by a matrix times the input. And we even know exactly what that matrix is:
Theorem
Let \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) be a linear transformation. Then \(T(\mathbf{x}) = A\mathbf{x}\) for all vectors \(\mathbf{x} \in \mathbb{R}^m\), where \(A\) is the \(n \times m\) matrix given by \[A = \begin{bmatrix} | & | & & | \\ T(\mathbf{e}_1) & T(\mathbf{e}_2) & \dotsm & T(\mathbf{e}_m) \\ | & | & & | \end{bmatrix}.\]
Because of this, we could say that matrices and linear transformations are equivalent concepts. Every matrix is secretly a linear transformation, and every linear transformation is secretly a matrix.
Every linear transformation \(T\) is uniquely determined by the images of the standard basis vectors \(\mathbf{e}_1, \dots, \mathbf{e}_m\). In other words, if we know the values of \(T(\mathbf{e}_1), \dots, T(\mathbf{e}_m)\), then we can compute \(T(\mathbf{x})\) for any \(\mathbf{x} \in \mathbb{R}^m\). Here are two more-or-less equivalent ways to see this:
Now let's see some examples of linear transformations and compute the corresponding matrices using what we just learned.
Example
Maybe the simplest example of a linear transformation is the identity map: this is \(T \colon \mathbb{R}^m \to \mathbb{R}^m\) (note that the domain and codomain are the same) given by \(T(\mathbf{x}) = \mathbf{x}.\) It's easy to check that this is a linear transformation. What is its matrix? We know that the \(i\)th column should be the image of \(\mathbf{e}_i\). But that is simply \(\mathbf{e}_i\) itself, so we see that the matrix of \(T\) will look like \[\begin{bmatrix} 1 & 0 & \cdots & 0 \\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \end{bmatrix},\] with \(1\)s going down the diagonal. (The first column of this matrix is \(\mathbf{e}_1\), the second is \(\mathbf{e}_2\), and so on.) This is called the \(m \times m\) identity matrix and is denoted \(I_m\). Notice that this is the only \(m \times m\) matrix in reduced echelon form with a pivot in every row/column.
Example
As a more interesting example, let \(\theta\) be any real number. Consider the function \(R_\theta \colon \mathbb{R}^2 \to \mathbb{R}^2\) that rotates any vector \(\mathbf{x} \in \mathbb{R}^2\) counterclockwise through an angle of \(\theta\) (radians). Maybe surprisingly, \(R_\theta\) is a linear transformation.
(To see why geometrically, remember that adding two vectors is equivalent to drawing the parallelogram determined by those vectors and taking the "long" diagonal; then rotating both vectors by \(\theta\) will also rotate the entire parallelogram by \(\theta\), so it rotates the sum by \(\theta\) as well. Also, it is clear that scaling a vector will also scale its rotation by the same amount.)
What is the matrix corresponding to \(R_\theta\)? As in the previous example, we want to know what \(R_\theta\) does to the standard basis vectors. There are only two in this case: \(\mathbf{e}_1 = \begin{bmatrix}1\\0 \end{bmatrix}\) and \(\mathbf{e}_2 = \begin{bmatrix}0\\1 \end{bmatrix}\). By doing a bit of trigonometry, we can get that \[ R_\theta (\mathbf{e}_1) = \begin{bmatrix} \cos(\theta) \\ \sin(\theta) \end{bmatrix} \qquad \text{and} \qquad R_\theta (\mathbf{e}_2) = \begin{bmatrix} -\sin(\theta) \\ \cos(\theta) \end{bmatrix}. \] Therefore, the matrix of \(R_\theta\) is the rotation matrix \[ \begin{bmatrix} \cos(\theta) & -\sin(\theta) \\ \sin(\theta) & \cos(\theta) \end{bmatrix}.\] As a warning, notice that the entries of this matrix are constants, only depending on the value of \(\theta\) we chose at the beginning. We never actually take the cosine or sine of (any component of) the input vector; that wouldn't be very linear!
For instance, for \(\theta = \pi/3\) (or 60 degrees), the rotation matrix is \[ \begin{bmatrix} \cos(\pi/3) & -\sin(\pi/3) \\ \sin(\pi/3) & \cos(\pi/3) \end{bmatrix} = \begin{bmatrix} 1/2 & -\sqrt{3}/2 \\ \sqrt{3}/2 & 1/2 \end{bmatrix}.\]
We now discuss two important properties that a linear transformation (or any function) can have: being one-to-one and being onto.
Definition
A linear transformation \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) is said to be one-to-one (or injective) if, for all \(\mathbf{y} \in \mathbb{R}^n\), there is at most one \(\mathbf{x} \in \mathbb{R}^m\) such that \(T(\mathbf{x}) = \mathbf{y}.\)
In other words, for a one-to-one function, every possible output comes from at most one input; different inputs map to different outputs. A good way to write the one-to-one condition is: "If \(T(\mathbf{x}) = T(\mathbf{y})\), then \(\mathbf{x} = \mathbf{y}\)." (If two inputs map to the same output, then the inputs must be the same.)
Example
The linear transformation \(T \colon \mathbb{R}^2 \to \mathbb{R}^2\) given by \(T\left(\begin{bmatrix} a \\ b \end{bmatrix}\right) = \begin{bmatrix} a+b \\ b \end{bmatrix}\) is one-to-one. To see why, suppose that \(T\left(\begin{bmatrix} a \\ b \end{bmatrix}\right) = T\left(\begin{bmatrix} c \\ d \end{bmatrix}\right).\) Then, \(\begin{bmatrix} a+b \\ b \end{bmatrix} = \begin{bmatrix} c+d \\ d \end{bmatrix}.\) This means that \(b = d\) from the second components being equal, and then the first components being equal gives \(a = c.\) Therefore, \(\begin{bmatrix} a\\ b \end{bmatrix} = \begin{bmatrix} c \\ d \end{bmatrix}.\)
Example
The linear transformation \(T \colon \mathbb{R}^3 \to \mathbb{R}^2\) given by \(T\left(\begin{bmatrix} a \\ b \\ c \end{bmatrix}\right) = \begin{bmatrix} a \\ b \end{bmatrix}\) is not one-to-one, because (for example) \(T\left(\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}\right)\) and \(T\left(\begin{bmatrix} 1 \\ 2 \\ 4 \end{bmatrix}\right)\) both equal \(\begin{bmatrix} 1 \\ 2 \end{bmatrix}.\)
For \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) to be one-to-one, we need that no two inputs ever give the same output. There are a ton of different kinds of linear transformations, but regardless of how it is defined, we always know one input-output pair:
Fact
If \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) is a linear transformation, then \(T(\mathbf{0}) = \mathbf{0}.\)
(Be careful: the first \(\mathbf{0}\) is in \(\mathbb{R}^m\), but the second is in \(\mathbb{R}^n\)!) To see this, we can either work out that any matrix times the zero vector is zero, or use the properties of linear transformations: for any \(c\), we have \(T(c \cdot \mathbf{0}) = c \cdot T(\mathbf{0})\), or \(T(\mathbf{0}) = c T(\mathbf{0}).\) In particular, when \(c = 0\) we get \(T(\mathbf{0}) = \mathbf{0}.\)
So, because \(T(\mathbf{0}) = \mathbf{0}\), we know that for \(T\) to be one-to-one, no other vector in \(\mathbb{R}^m\) can be mapped to \(\mathbf{0}.\) However, it turns out that this is also a sufficient condition:
Fact
A linear transformation \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) is one-to-one if and only if the equation \(T(\mathbf{x}) = \mathbf{0}\) has the unique solution \(\mathbf{x} = \mathbf{0}.\)
(In other words, \(T\) is one-to-one if and only if \(T(\mathbf{x}) = \mathbf{0}\) implies \(\mathbf{x} = \mathbf{0}.\))
We already know that, if \(T\) is one-to-one, then \(T(\mathbf{x}) = \mathbf{0}\) must only have the solution \(\mathbf{x} = \mathbf{0}.\) Why is the opposite true? To see why, let's suppose that \(T\) is not one-to-one, and prove that \(T(\mathbf{x}) = \mathbf{0}\) has a solution besides \(\mathbf{x} = \mathbf{0}.\) If \(T\) is not one-to-one, then some two different inputs map to the same output; suppose \(T(\mathbf{x}) = T(\mathbf{x'}) = \mathbf{y}.\) Now we can use the properties of linear transformations to write the following: \[T(\mathbf{x} - \mathbf{x'}) = T(\mathbf{x}) - T(\mathbf{x'}) = \mathbf{y} - \mathbf{y} = \mathbf{0}.\] Since \(\mathbf{x}\) and \(\mathbf{x'}\) are different, \(\mathbf{x} - \mathbf{x'}\) is nonzero, so we've shown that \(T\) maps some nonzero vector to zero, which is what we wanted.
Now let's apply this to the corresponding matrix \(A\), to get that \(T\) is one-to-one if and only if the equation \(A\mathbf{x} = \mathbf{0}\) has the unique solution \(\mathbf{x} = \mathbf{0}.\) Hey, that looks familiar: it's exactly the condition for linear independence of the columns of \(A\)!
Theorem
Let \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) be a linear transformation and let \(A\) be its matrix. Then, \(T\) is one-to-one if and only if the columns of \(A\) are linearly independent.
As a consequence, if \(m > n\), then a linear transformation \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) can never be one-to-one. (Intuitively: if \(m > n\), then \(\mathbb{R}^m\) is "bigger" than \(\mathbb{R}^n\), so there's no linear transformation that maps \(\mathbb{R}^m\) onto \(\mathbb{R}^n\) without mapping two points in \(\mathbb{R}^m\) to the same point in \(\mathbb{R}^n.\))