Our starting point for exploring linear algebra will be solving systems of linear equations. Here is an example of a system of linear equations:
\(\left\{ \begin{align*} x_1 + 2x_2 &= 5 \\ 3x_1 + 4x_2 &= 11 \end{align*} \qquad (*) \right.\)
Before we try to solve it, let's define some terms. What exactly does "solving" a "system" of "linear equations" mean?
A linear equation is an equation that has the form
\[a_1x_1 + a_2x_2 + \dotsm + a_nx_n = b.\]
Here, \(x_1, x_2, \dots, x_n\) are the variables, and \(a_1, a_2, \dots, a_n, b\) are constants. A linear equation can involve any number of variables; in this case, the number of variables is represented by \(n\). However, we are only allowed to multiply the variables by constants and add them together.
A solution of a linear equation is a choice of values (real numbers) for the variables \(x_1, x_2, \dots, x_n\) such that the equation becomes true when we substitute in those values.
Example
Consider the first equation from \((*)\) above: \(x_1 + 2x_2 = 5\). Is \((x_1, x_2) = (3, 1)\) a solution to this equation? To check, we substitute the corresponding values for \(x_1\) and \(x_2\) and calculate:
\[x_1 + 2x_2 = 3 + 2(1) = 5\]
The equation therefore becomes \(5 = 5\), which is true, so \((x_1, x_2) = (3, 1)\) is a solution of the equation. Hooray!
A system of linear equations is a list of linear equations involving the same variables. A solution of a system of linear equations is a choice of values for the variables such that all linear equations become true when we substitute in those values. In other words, we need all the equations to be true simultaneously. Finally, solving a linear equation or system of linear equations just means finding all its solutions.
Example
In our example \((*)\), there are two linear equations and two variables. We know that \((x_1, x_2) = (3, 1)\) is a solution for just the first equation. Is it also a solution to the second equation? Well, if \(x_1=3\) and \(x_2=1\), then
\[3x_1 + 4x_2 = 3(3) + 4(1) = 13\]
The second equation becomes \(13 = 11\), which isn't true! So unfortunately, \((x_1, x_2) = (3, 1)\) is not a solution to the system of equations.
In general, a system of linear equations will look like this:
\[\left\{ \begin{align*} a_{1,1}x_1 + a_{1,2}x_2 + \dotsm + a_{1,n} x_n &= b_1 \\ a_{2,1}x_1 + a_{2,2}x_2 + \dotsm + a_{2,n} x_n &= b_2 \\ &\vdots \\ a_{m,1}x_1 + a_{m,2}x_2 + \dotsm + a_{m,n} x_n &= b_m \end{align*} \right.\]
This system has \(m\) equations and \(n\) variables. The variables are named \(x_1, x_2, \dots, x_n\), and all the \(a_{i,j}\)'s and \(b_i\)'s are constants. (Notice that \(m\) and \(n\) do not need to be the same! In fact, we will be able to say a lot about the set of solutions of a system of equations, based only on the number of equations and the number of variables.)
Now that we know what we're trying to do, let's do it! Let's try to solve our system of linear equations from above:
\(\left\{ \begin{align*} x_1 + 2x_2 &= 5 \\ 3x_1 + 4x_2 &= 11 \end{align*} \right.\)
There are two main ways to solve a system of linear equations by hand: substitution and elimination. Let's take a look at both.
The substitution method
In the first equation, we can rearrange the terms in order to solve for \(x_1\) in terms of \(x_2\), which gives
\(x_1 = -2x_2 + 5.\)
Now that we have a formula for \(x_1\) in terms of \(x_2\), we can substitute this formula into the second equation, to get an equation only involving \(x_2\):
\(3(-2x_2+5) + 4x_2 = 11.\)
This is great, because we know how to solve linear equations with only one variable! Simplifying the left-hand side, we get \(-2x_2 + 15 = 11\). Then, \(-2x_2 = -4\), so \(x_2 = 2\).
We've found the value of \(x_2\), but our solution needs to have a value for both our variables. Luckily, we already have a formula that tells us the value of \(x_1\) given the value of \(x_2\): we have
\(x_1 = -2x_2 + 5 = -2(2) + 5 = 1.\)
Therefore, this system of linear equations has a unique (meaning exactly one) solution: \((x_1, x_2) = (1, 2)\). (If we want to be sure that this is actually a solution, we can simply substitute these values into our original system and check that both of the equations become true.)
Because this system of equations has a solution, we say that the system is consistent. In general, any system of equations with at least one solution is called consistent. We will see later that some systems can have no solutions; we call these inconsistent.
The substitution method certainly works. However, for the purposes of setting up the theory of linear algebra, we will actually prefer to use and think about the elimination method:
The elimination method
The goal of this method is also to take our system of two equations in two variables, and produce a single equation in only one variable. However, in the elimination method, we don't solve for one variable in terms of the others. Instead, we "scale" and add our equations together in a way that eliminates variables. Here is how it works. In our example, let's multiply the first equation by \(-3\), to get
\[\left\{ \begin{align*} -3x_1 - 6x_2 &= -15 \\ 3x_1 + 4x_2 &= 11 \end{align*} \right.\]
Why did we do this? Well, both of our equations still have to be true for any solution to the system. So, the sum of the equations must also be true. That is, the sum of the left-hand sides must equal the sum of the right-hand sides. If we sum up the equations in this way, we get
\[-2x_2 = -4\]
which is an equation in only one variable! That's why we multiplied by \(-3\): in order to have the \(x_1\) terms cancel out when we did this sum. Now we can solve for \(x_2\) like in the substitution method, and then we can use our original first equation, \(x_1 + 2x_2 = 5\), to solve for \(x_1\).
In the elimination method, the key step is taking one equation and adding a multiple of another equation to it. In the example above, we essentially went from
\[\left\{ \begin{align*} x_1 + 2x_2 &= 5 \\ 3x_1 + 4x_2 &= 11 \end{align*} \right.\]
to
\(\left\{ \begin{align*} x_1 + 2x_2 &= 5 \\ -2x_2 &= -4 \end{align*} \right.\)
by adding \(-3\) times the first equation to the second equation. This operation preserves the set of solutions to the system, meaning that any solution to the first system will also be a solution to the second system, and vice versa. (Think about why this is true!) By performing these steps repeatedly, we can simplify the system until the solutions are "easy" to see. This process is especially helpful in larger systems (with more than 2 equations and 2 variables).
In this class (and in math in general!), it's important to try to visualize what we are doing geometrically. In our case, a linear equation with two variables defines a line when graphed in the plane (in \(\mathbb{R}^2\)). Then, solving a system of two linear equations in two variables is equivalent to finding the intersection of the lines. (Please see lecture recordings for some drawings.)
In general, a linear equation in \(n\) variables will define a hyperplane: a line for \(n=2\), a plane for \(n=3\), and similar higher-dimensional analogs.
Let's try two more illustrative examples:
Example
Here is another system of linear equations in two variables:
\[\left\{ \begin{align*} 2x_1 - x_2 &= -3 \\ -4x_1 + 2x_2 &= 6 \end{align*} \right.\]
Let's try to solve this system using elimination. In order to eliminate \(x_1\) from the second equation, we should multiply the first equation by \(2\), to get \(4x_1 - 2x_2 = -6\), and add that to the second equation. The result is:
\[\left\{ \begin{align*} 2x_1 - x_2 &= -3 \\ 0 &= 0 \end{align*} \right.\]
What does this mean? Well, this system has the same solution set as the previous one. But the second equation doesn't have any variables in it at all; in fact, it's just a true statement! In other words, the system we started with is equivalent to just the single equation \(2x_1 - x_2 = -3\).
This equation has infinitely many solutions for \((x_1, x_2)\). However, we can still describe the solution set, as follows. For any value of \(x_1\), say \(x_1 = t\), we can substitute and then solve for \(x_2\), to get \(x_2 = 2t+3\). Therefore, the solutions to this system are all pairs of the form \((x_1, x_2) = (t, 2t+3)\), where \(t\) can be any real number.
Geometrically speaking, this system corresponds to two lines that coincide (i.e., are the exact same).
Example
What happens if we change the numbers slightly in the previous example? Let's consider the system
\[\left\{ \begin{align*} 2x_1 - x_2 &= -3 \\ -4x_1 + 2x_2 &= 7 \end{align*} \right.\]
(We changed the \(6\) on the right-hand side of the second equation to \(7\).) In using the elimination method, we should again add two times the first equation to the second equation. This time, the result is:
\(\left\{ \begin{align*} 2x_1 - x_2 &= -3 \\ 0 &= 1 \end{align*} \right.\)
Again, the second equation does not have any variables in it, but now it is a false statement! This means that this system of equations has no solutions: if there were some values for \(x_1, x_2\) that satisfied the system, then by using elimination as we just did, we could derive a false statement, \(0=1\). In other words, this is an inconsistent system.
Geometrically speaking, this system corresponds to two parallel lines (which, of course, don't intersect).
As we will see, these three examples represent all the possibilities for the solution set of a system of linear equations:
Theorem (Solution Sets of Systems of Linear Equations)
Every system of linear equations either has a unique solution (consistent), infinitely many solutions (consistent), or no solutions (inconsistent).