Exercise
Midterm review! For each entry in this table, either find an example of vectors \(\mathbf{v}_1, \dots, \mathbf{v}_m \in \mathbb{R}^n\) satisfying the corresponding criteria, or explain why no example exists:
Vectors with certain properties Properties don't span \(\mathbb{R}^n\), linearly dependent don't span \(\mathbb{R}^n\), linearly independent span \(\mathbb{R}^n\), linearly dependent span \(\mathbb{R}^n\), linearly independent \(m<n\) \(m=n\) \(m>n\)
Today we'll begin to talk about linear transformations (Chapter 3), which are certain "nice" functions whose input and output are both vectors. As we will see, linear transformations give us yet another way to interpret systems of equations and matrices, and can also be viewed geometrically. They also appear very frequently in economics, statistics, computer science, data analysis, and just about any other field that uses linear algebra.
As you will recall, a function is a rule \(f\) that takes as input an element from some set, called the domain of the function, and produces an element from a second set, called the codomain of the function. We write \(f \colon S \to T\) to denote a function \(f\) whose domain is the set \(S\) and whose codomain is the set \(T\). For an element \(s \in S\), the image of \(s\) under \(f\) is the element of \(T\), denoted \(f(s)\), that \(s\) is mapped to by \(f\).
A function must be defined for every element in its domain. However, a function does not necessarily need to "hit" every element in its codomain. The range of a function is the set of all elements that are the image of some element in its domain. Therefore, the range of a function is a subset of its codomain. (It can be the same as the codomain, or smaller.)
Example
Consider the function \(f \colon \mathbb{R} \to \mathbb{R}\) defined by \(f(x) = x^2\). This function has domain \(\mathbb{R}\) and codomain \(\mathbb{R}\). However, the range of \(f\) is not all of \(\mathbb{R}\), because the square of any real number is always nonnegative. In fact, the range of \(f\) is the set of all nonnegative real numbers (sometimes denoted \(\mathbb{R}_{\ge 0}\).
Example
Consider the sine function, \(\sin \colon \mathbb{R} \to \mathbb{R}\). Again, the domain and codomain are both \(\mathbb{R}\), but as you may recall, the range of the sine function is just the closed interval \([-1, 1]\).
In this class, we will be most interested in functions that map vectors to vectors. In other words, we will mainly study functions of the form \[f \colon \mathbb{R}^m \to \mathbb{R}^n\] for some \(m\) and \(n\).
Example
We can define a function \(f \colon \mathbb{R}^2 \to \mathbb{R}^3\) by \[ f \left(\begin{bmatrix} a \\ b \end{bmatrix}\right) = \begin{bmatrix} a+b \\ a-b \\ 20a \end{bmatrix}. \] Then, \(f \left(\begin{bmatrix} 2 \\ 1 \end{bmatrix}\right) = \begin{bmatrix} 3 \\ 1 \\ 40 \end{bmatrix}\).
Because this is linear algebra, we are especially interested in functions that are "compatible" with the nice features of vectors we already know, namely that we can add them and multiply them by scalars. Such functions are called linear transformations (or sometimes just linear maps):
Definition
A function \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) is a linear transformation if it satisfies both of the following properties:
(i) \(T\) respects addition: for all vectors \(\mathbf{v}, \mathbf{w} \in \mathbb{R}^m\), we have \(T(\mathbf{v} + \mathbf{w}) = T(\mathbf{v}) + T(\mathbf{w})\).
(ii) \(T\) respects scalar multiplication: for all vectors \(\mathbf{v} \in \mathbb{R}^m\) and all scalars \(c \in \mathbb{R}^m\), we have \(T(c\mathbf{v}) = c \cdot T(\mathbf{v})\).
("T" is for Transformation.) In other words, if \(T\) is a linear transformation, then applying \(T\) to the sum of two vectors is the same as applying \(T\) to each vector individually and then adding the results; and applying \(T\) to a scalar multiple of a vector is the same as applying \(T\) to the original vector and then scaling the result (by the same amount).
Example
Above, we defined a function \(f \colon \mathbb{R}^2 \to \mathbb{R}^3\) by \[ f \begin{bmatrix} a \\ b \end{bmatrix} = \begin{bmatrix} a+b \\ a-b \\ 20a \end{bmatrix}. \] Let's show that \(f\) is a linear transformation; we just need to check that it satisfies properties (i) and (ii).
(i) We need to check that for all \(\mathbf{v}, \mathbf{w} \in \mathbb{R}^2\), we have \[f(\mathbf{v} + \mathbf{w}) = f(\mathbf{v}) + f(\mathbf{w}).\] To do this, let \(\mathbf{v} = \begin{bmatrix}v_1\\v_2\end{bmatrix}\) and \(\mathbf{w} = \begin{bmatrix}w_1\\w_2\end{bmatrix}\) where \(v_1, v_2, w_1, w_2\) are some real numbers. Then we calculate \(f(\mathbf{v}+\mathbf{w})\) and \(f(\mathbf{v}) + f(\mathbf{w})\) and make sure we get the same thing: \[f(\mathbf{v}+\mathbf{w}) = f\left( \begin{bmatrix}v_1\\v_2\end{bmatrix} + \begin{bmatrix}w_1\\w_2\end{bmatrix} \right) = f\left(\begin{bmatrix}v_1+w_1\\v_2+w_2\end{bmatrix}\right) = \begin{bmatrix}(v_1+w_1)+(v_2+w_2)\\(v_1+w_1)-(v_2+w_2)\\20(v_1+w_1)\end{bmatrix} \] and \[ f(\mathbf{v}) + f(\mathbf{w}) = f\left(\begin{bmatrix}v_1\\v_2\end{bmatrix}\right) + f\left(\begin{bmatrix}w_1\\w_2\end{bmatrix}\right) = \begin{bmatrix} v_1+v_2 \\ v_1-v_2 \\ 20v_1\end{bmatrix} + \begin{bmatrix} w_1+w_2 \\ w_1-w_2 \\ 20w_1\end{bmatrix} = \begin{bmatrix} v_1+v_2 + w_1+w_2 \\ v_1-v_2 + w_1 - w_2 \\ 20v_1 + 20w_1\end{bmatrix}.\] These are the same, so our function \(f\) does satisfy property (i).
For property (ii), we can follow the same procedure. Given any scalar \(c\) and any vector \(\mathbf{v} = \begin{bmatrix}v_1\\v_2\end{bmatrix}\), we compute: \[f(c \mathbf{v}) = f\left(c \begin{bmatrix}v_1\\v_2\end{bmatrix} \right) = f\left(\begin{bmatrix}cv_1\\cv_2\end{bmatrix} \right) = \begin{bmatrix} cv_1 + cv_2 \\ cv_1 - cv_2 \\ 20cv_1 \end{bmatrix}\] and \[c \cdot f(\mathbf{v}) = c \cdot f\left(\begin{bmatrix}v_1\\v_2\end{bmatrix}\right) = c \cdot \begin{bmatrix} v_1+v_2 \\ v_1-v_2 \\ 20v_1\end{bmatrix} = \begin{bmatrix} c(v_1+v_2) \\ c(v_1-v_2) \\ c \cdot 20v_1\end{bmatrix}.\] Again, these are the same, so \(f\) satisfies property (ii).
Therefore, \(f\) is indeed a linear transformation.
This kind of computation is the standard way to prove that a function is a linear transformation. However, we will soon see a more convenient way to do this. For now, let's consider a non-example of a linear transformation:
Non-Example
Consider the function \(f \colon \mathbb{R}^3 \to \mathbb{R}^3 \) defined by \[ f \left( \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} \right) = \begin{bmatrix} x_1^2 \\ x_2 \\ x_1 + x_3 \end{bmatrix}.\] As it turns out, this is not a linear transformation. (You might suspect as much because the definition involves squaring one of the variables.) Let's see why.
To show that a function is not a linear transformation, we only need to find one case where either property (i) or (ii) does not hold. There are usually many such cases: here, let's try vectors \(\mathbf{v} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}\) and \(\mathbf{w} = \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix}\). On one hand, we have \(\mathbf{v}+\mathbf{w} = \begin{bmatrix}5 \\ 7 \\ 9 \end{bmatrix}\), so \[f(\mathbf{v}+\mathbf{w}) = \begin{bmatrix} 25 \\ 7 \\ 14 \end{bmatrix}.\] On the other hand, we have \[f(\mathbf{v}) + f(\mathbf{w}) = \begin{bmatrix} 1 \\ 2 \\ 4 \end{bmatrix} + \begin{bmatrix} 16 \\ 5 \\ 10 \end{bmatrix} = \begin{bmatrix} 17 \\ 7 \\ 14 \end{bmatrix},\] which is not the same. This means that \(f(\mathbf{v}+\mathbf{w}) \neq f(\mathbf{v})+f(\mathbf{w})\) in this case, so \(f\) is not a linear transformation.
Another way to think of linear transformations is that they are functions which respect linear combinations. That is, \(T \colon \mathbb{R}^m \to \mathbb{R}^n \) is a linear transformation if and only if, for all vectors \(\mathbf{v}, \mathbf{w} \in \mathbb{R}^m\) and all scalars \(c, d \in \mathbb{R}\), we have \[T(c\mathbf{v} + d\mathbf{w}) = c \cdot T(\mathbf{v}) + d \cdot T(\mathbf{w}).\] This is essentially "combining" properties (i) and (ii) in the definition of linear transformation into one property that encompasses them both.
We now explore an essential way that linear transformations are connected to what we've studied so far. Specifically, every matrix corresponds to a linear transformation. Here's what we mean:
Fact
Suppose \(A\) is any \(n \times m\) matrix. Define a function \(T \colon \mathbb{R}^m \to \mathbb{R}^n\) by \[T(\mathbf{x}) = A\mathbf{x}\] for vectors \(\mathbf{x} \in \mathbb{R}^n.\) Then \(T\) is a linear transformation.
In other words, any function that multiplies a vector in \(\mathbb{R}^m\) by some fixed matrix is a linear transformation. To see why this is true, we just have to remember how we defined \(A\mathbf{x}\) in the first place: if the columns of \(A\) are \(\mathbf{v}_1, \dots, \mathbf{v}_m\) and \(\mathbf{x} = (x_1, \dots, x_m)\), then \[ A\mathbf{x} = x_1\mathbf{v}_1 + \dotsm + x_m \mathbf{v}_m. \] Then, properties (i) and (ii) in the definition of linear transformation will be true because of how vector arithmetic works. For property (i), if we also have a vector \(\mathbf{y} = (y_1, \dots, y_m)\), then \[T(\mathbf{x}+\mathbf{y}) = A(\mathbf{x}+\mathbf{y}) = (x_1+y_1)\mathbf{v}_1 + \dotsm + (x_2+y_2)\mathbf{v}_m,\] while \[T(\mathbf{x})+T(\mathbf{y}) = A\mathbf{x}+A\mathbf{y} = (x_1\mathbf{v}_1 + \dotsm + x_m\mathbf{v}_m) + (y_1\mathbf{v}_1 + \dotsm + y_m\mathbf{v}_m),\] which are equal using the commutativity and distributivity properties for vectors. Property (ii) can be checked in a similar way.
Note: Because of how matrix-vector multiplication works, in order to get a linear transformation from \(\mathbb{R}^m\) to \(\mathbb{R}^n\), we need the matrix to have dimensions \(n \times m,\) not \(m \times n\). This can be annoying but is really important to remember!
This gives us an easier way to show that a function is a linear transformation: if we can re-write it as a matrix times the input, then we know automatically that it is a linear transformation (we already did the work to prove that any such function is a linear transformation!).
Example
Let's return to our function \(f \colon \mathbb{R}^2 \to \mathbb{R}^3\) defined by \[ f \begin{bmatrix} a \\ b \end{bmatrix} = \begin{bmatrix} a+b \\ a-b \\ 20a \end{bmatrix}. \] Using some algebra, we can write \[ f \begin{bmatrix} a \\ b \end{bmatrix} = \begin{bmatrix} a+b \\ a-b \\ 20a \end{bmatrix} = \begin{bmatrix} a \\ a \\ 20a \end{bmatrix} + \begin{bmatrix} b \\ -b \\ 0 \end{bmatrix} = a \begin{bmatrix} 1 \\ 1 \\ 20 \end{bmatrix} + b \begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix} = \begin{bmatrix} 1 & 1 \\ 1 & -1 \\ 20 & 0 \end{bmatrix} \begin{bmatrix} a \\ b \end{bmatrix}.\] Therefore, \(f\) is of the form \(f(\mathbf{x}) = A\mathbf{x}\), where \(A\) is the matrix \(\begin{bmatrix} 1 & 1 \\ 1 & -1 \\ 20 & 0 \end{bmatrix}\), so \(f\) is a linear transformation.
We just showed that every matrix corresponds to a linear transformation, where the linear transformation is defined by multiplying by the matrix. But how about the reverse question? Does every linear transformation correspond to a matrix? Or are there "weird" linear transformations that cannot be written as a matrix times the input?
We'll see the answer on Monday after the midterm.