[next] [prev] [prev-tail] [tail] [up]

### 3.2Matrices and linear systems

Note: 1 and a half lectures, ﬁrst part of §5.1 in [EP], §7.2 and §7.3 in [BD]

#### 3.2.1Matrices and vectors

Before we start talking about linear systems of ODEs, we need to talk about matrices, so let us review these brieﬂy. A matrix is an array of numbers ( rows and columns). For example, we denote a matrix as follows

By a vector we usually mean a column vector, that is an matrix. If we mean a row vector we will explicitly say so (a row vector is a matrix). We usually denote matrices by upper case letters and vectors by lower case letters with an arrow such as or . By we mean the vector of all zeros.

We deﬁne some operations on matrices. We want matrices to really act like numbers, so our operations have to be compatible with this viewpoint.

First, we can multiply a matrix by a scalar (a number). We simply multiply each entry in the matrix by the scalar. For example,

Matrix addition is also easy. We add matrices element by element. For example,

If the sizes do not match, then addition is not deﬁned.

If we denote by 0 the matrix with all zero entries, by , scalars, and by , , matrices, we have the following familiar rules:

Another useful operation for matrices is the so-called transpose. This operation just swaps rows and columns of a matrix. The transpose of is denoted by . Example:

#### 3.2.2Matrix multiplication

Let us now deﬁne matrix multiplication. First we deﬁne the so-called dot product (or inner product) of two vectors. Usually this will be a row vector multiplied with a column vector of the same size. For the dot product we multiply each pair of entries from the ﬁrst and the second vector and we sum these products. The result is a single number. For example,

And similarly for larger (or smaller) vectors.

Armed with the dot product we deﬁne the product of matrices. First let us denote by the row of and by the column of . For an matrix and an matrix we can deﬁne the product . We let be an matrix whose entry is the dot product

Do note how the sizes match up: multiplied by is . Example:

For multiplication we want an analogue of a 1. This analogue is the so-called identity matrix. The identity matrix is a square matrix with 1s on the main diagonal and zeros everywhere else. It is usually denoted by . For each size we have a diﬀerent identity matrix and so sometimes we may denote the size as a subscript. For example, the would be the identity matrix

We have the following rules for matrix multiplication. Suppose that , , are matrices of the correct sizes so that the following make sense. Let denote a scalar (number).

A few warnings are in order.

(i)
in general (it may be true by ﬂuke sometimes). That is, matrices do not commute. For example, take and .
(ii)
does not necessarily imply , even if is not 0.
(iii)
does not necessarily mean that or . For example, take .

For the last two items to hold we would need to “divide” by a matrix. This is where the matrix inverse comes in. Suppose that and are matrices such that

Then we call the inverse of and we denote by . If the inverse of exists, then we call invertible. If is not invertible we sometimes say is singular.

If is invertible, then does imply that (in particular the inverse of is unique). We just multiply both sides by (on the left) to get or or . It is also not hard to see that .

#### 3.2.3The determinant

For square matrices we deﬁne a useful quantity called the determinant. We deﬁne the determinant of a matrix as the value of its only entry. For a matrix we deﬁne

Before trying to compute the determinant for larger matrices, let us note the meaning of the determinant. Consider an matrix as a mapping of the dimensional euclidean space to itself, where gets sent to . In particular, a matrix is a mapping of the plane to itself. The determinant of is the factor by which the area of objects gets changed. If we take the unit square (square of side 1) in the plane, then takes the square to a parallelogram of area . The sign of denotes changing of orientation (negative if the axes get ﬂipped). For example, let

Then . Let us see where the square with vertices , , , and gets sent. Clearly gets sent to .

The image of the square is another square with vertices , , , and . The image square has a side of length and is therefore of area 2.

If you think back to high school geometry, you may have seen a formula for computing the area of a parallelogram with vertices , , and . And it is precisely

The vertical lines above mean absolute value. The matrix carries the unit square to the given parallelogram.

Let us deﬁne the determinant for larger matrices. We deﬁne as the matrix with the row and the column deleted. To compute the determinant of a matrix, pick one row, say the row and compute:

For the ﬁrst row we get

We alternately add and subtract the determinants of the submatrices for a ﬁxed and all . For a matrix, picking the ﬁrst row, we get . For example,

The numbers are called cofactors of the matrix and this way of computing the determinant is called the cofactor expansion. It is also possible to compute the determinant by expanding along columns (picking a column instead of a row above).

Note that a common notation for the determinant is a pair of vertical lines:

I personally ﬁnd this notation confusing as vertical lines usually mean a positive quantity, while determinants can be negative. I will not use this notation in this book.

One of the most important properties of determinants (in the context of this course) is the following theorem.

Theorem 3.2.1. An matrix is invertible if and only if .

In fact, there is a formula for the inverse of a matrix

Notice the determinant of the matrix in the denominator of the fraction. The formula only works if the determinant is nonzero, otherwise we are dividing by zero.

#### 3.2.4Solving linear systems

One application of matrices we will need is to solve systems of linear equations. This is best shown by example. Suppose that we have the following system of linear equations

Without changing the solution, we could swap equations in this system, we could multiply any of the equations by a nonzero number, and we could add a multiple of one equation to another equation. It turns out these operations always suﬃce to ﬁnd a solution.

It is easier to write the system as a matrix equation. The system above can be written as

To solve the system we put the coeﬃcient matrix (the matrix on the left hand side of the equation) together with the vector on the right and side and get the so-called augmented matrix

We apply the following three elementary operations.

(i)
Swap two rows.
(ii)
Multiply a row by a nonzero number.
(iii)
Add a multiple of one row to another row.

We keep doing these operations until we get into a state where it is easy to read oﬀ the answer, or until we get into a contradiction indicating no solution, for example if we come up with an equation such as .

Let us work through the example. First multiply the ﬁrst row by to obtain

Now subtract the ﬁrst row from the second and third row.

Multiply the last row by and the second row by .

Swap rows 2 and 3.

Subtract the last row from the ﬁrst, then subtract the second row from the ﬁrst.

If we think about what equations this augmented matrix represents, we see that , , and . We try this solution in the original system and, voilà, it works!

Exercise 3.2.1: Check that the solution above really solves the given equations.

If we write this equation in matrix notation as

where is the matrix and is the vector . The solution can be also computed via the inverse,

One last note to make about linear systems of equations is that it is possible that the solution is not unique (or that no solution exists). It is easy to tell if a solution does not exist. If during the row reduction you come up with a row where all the entries except the last one are zero (the last entry in a row corresponds to the right hand side of the equation) the system is inconsistent and has no solution. For example for a system of 3 equations and 3 unknowns, if you ﬁnd a row such as in the augmented matrix, you know the system is inconsistent.

You generally try to use row operations until the following conditions are satisﬁed. The ﬁrst nonzero entry in each row is called the leading entry.

(i)
There is only one leading entry in each column.
(ii)
All the entries above and below a leading entry are zero.
(iii)

Such a matrix is said to be in reduced row echelon form. The variables corresponding to columns with no leading entries are said to be free variables. Free variables mean that we can pick those variables to be anything we want and then solve for the rest of the unknowns.

Example 3.2.1: The following augmented matrix is in reduced row echelon form.

Suppose the variables are , , and . Then is the free variable, , and .

On the other hand if during the row reduction process you come up with the matrix

there is no need to go further. The last row corresponds to the equation , which is preposterous. Hence, no solution exists.

#### 3.2.5Computing the inverse

If the matrix is square and there exists a unique solution to for any (there are no free variables), then is invertible. Multiplying both sides by , you can see that . So it is useful to compute the inverse if you want to solve the equation for many diﬀerent right hand sides .

We have a formula for the inverse, but it is also not hard to compute inverses of larger matrices. While we will not have too much occasion to compute inverses for larger matrices than by hand, let us touch on how to do it. Finding the inverse of is actually just solving a bunch of linear equations. If we can solve where is the vector with all zeros except a 1 at the position, then the inverse is the matrix with the columns for (exercise: why?). Therefore, to ﬁnd the inverse we write a larger augmented matrix , where is the identity matrix. We then perform row reduction. The reduced row echelon form of will be of the form if and only if is invertible. We then just read oﬀ the inverse .

#### 3.2.6Exercises

Exercise 3.2.2: Solve by using matrix inverse.

Exercise 3.2.3: Compute determinant of .

Exercise 3.2.4: Compute determinant of . Hint: Expand along the proper row or column to make the calculations simpler.

Exercise 3.2.5: Compute inverse of .

Exercise 3.2.6: For which is not invertible? Is there only one such ? Are there several? Inﬁnitely many?

Exercise 3.2.7: For which is not invertible? Find all such .

Exercise 3.2.8: Solve .

Exercise 3.2.9: Solve .

Exercise 3.2.10: Solve .

Exercise 3.2.11: Find 3 nonzero matrices , , and such that but .

Exercise 3.2.101: Compute determinant of

Exercise 3.2.102: Find such that is not invertible.

Exercise 3.2.103: Solve .

Exercise 3.2.104: Suppose are nonzero numbers. Let , . a) Compute . b) Compute .