###
Subsection 3.7.1 Geometric multiplicity

Take the diagonal matrix

\begin{equation*}
A =
\begin{bmatrix}
3 & 0 \\ 0 & 3
\end{bmatrix} .
\end{equation*}

\(A\) has an eigenvalue 3 of multiplicity 2. We call the multiplicity of the eigenvalue in the characteristic equation the *algebraic multiplicity*. In this case, there also exist 2 linearly independent eigenvectors, \(\left[ \begin{smallmatrix} 1 \\ 0 \end{smallmatrix} \right]\) and \(\left[ \begin{smallmatrix} 0 \\ 1 \end{smallmatrix} \right]\) corresponding to the eigenvalue 3. This means that the so-called *geometric multiplicity* of this eigenvalue is also 2.

In all the theorems where we required a matrix to have \(n\) distinct eigenvalues, we only really needed to have \(n\) linearly independent eigenvectors. For example, \({\vec{x}\,}' = A\vec{x}\) has the general solution

\begin{equation*}
\vec{x} =
c_1 \begin{bmatrix} 1 \\ 0 \end{bmatrix} e^{3t}
+ c_2 \begin{bmatrix} 0 \\ 1 \end{bmatrix} e^{3t} .
\end{equation*}

Let us restate the theorem about real eigenvalues. In the following theorem we will repeat eigenvalues according to (algebraic) multiplicity. So for the above matrix \(A\text{,}\) we would say that it has eigenvalues 3 and 3.

######
Theorem 3.7.1

Suppose the \(n \times n\) matrix \(P\) has \(n\) real eigenvalues (not necessarily distinct), \(\lambda_1\text{,}\) \(\lambda_2\text{,}\) , \(\lambda_n\text{,}\) and there are \(n\) linearly independent corresponding eigenvectors \(\vec{v}_1\text{,}\) \(\vec{v}_2\text{,}\) , \(\vec{v}_n\text{.}\) Then the general solution to \({\vec{x}\,}' = P\vec{x}\) can be written as

\begin{equation*}
\vec{x} = c_1 \vec{v}_1 e^{\lambda_1 t} +
c_2 \vec{v}_2 e^{\lambda_2 t} + \cdots +
c_n \vec{v}_n e^{\lambda_n t} .
\end{equation*}

The *geometric multiplicity* of an eigenvalue of algebraic multiplicity \(n\) is equal to the number of corresponding linearly independent eigenvectors. The geometric multiplicity is always less than or equal to the algebraic multiplicity. The theorem handles the case when these two multiplicities are equal for all eigenvalues. If for an eigenvalue the geometric multiplicity is equal to the algebraic multiplicity, then we say the eigenvalue is *complete*.

In other words, the hypothesis of the theorem could be stated as saying that if all the eigenvalues of \(P\) are complete, then there are \(n\) linearly independent eigenvectors and thus we have the given general solution.

If the geometric multiplicity of an eigenvalue is 2 or greater, then the set of linearly independent eigenvectors is not unique up to multiples as it was before. For example, for the diagonal matrix \(A =
\left[ \begin{smallmatrix} 3 & 0 \\ 0 & 3 \end{smallmatrix} \right]\) we could also pick eigenvectors \(\left[ \begin{smallmatrix} 1 \\ 1 \end{smallmatrix} \right]\) and \(\left[ \begin{smallmatrix} 1 \\ -1 \end{smallmatrix} \right]\text{,}\) or in fact any pair of two linearly independent vectors. The number of linearly independent eigenvectors corresponding to \(\lambda\) is the number of free variables we obtain when solving \(A\vec{v} =
\lambda \vec{v}\text{.}\) We pick specific values for those free variables to obtain eigenvectors. If you pick different values, you may get different eigenvectors.

###
Subsection 3.7.2 Defective eigenvalues

If an \(n \times n\) matrix has less than \(n\) linearly independent eigenvectors, it is said to be *deficient*. Then there is at least one eigenvalue with an algebraic multiplicity that is higher than its geometric multiplicity. We call this eigenvalue *defective* and the difference between the two multiplicities we call the *defect*.

######
Example 3.7.1

The matrix

\begin{equation*}
\begin{bmatrix}
3 & 1 \\ 0 & 3
\end{bmatrix}
\end{equation*}

has an eigenvalue 3 of algebraic multiplicity 2. Let us try to compute eigenvectors.

\begin{equation*}
\begin{bmatrix}
0 & 1 \\ 0 & 0
\end{bmatrix}
\begin{bmatrix}
v_1 \\ v_2
\end{bmatrix}
= \vec{0} .
\end{equation*}

We must have that \(v_2 = 0\text{.}\) Hence any eigenvector is of the form \(\left[ \begin{smallmatrix} v_1 \\ 0 \end{smallmatrix} \right]\text{.}\) Any two such vectors are linearly dependent, and hence the geometric multiplicity of the eigenvalue is 1. Therefore, the defect is 1, and we can no longer apply the eigenvalue method directly to a system of ODEs with such a coefficient matrix.

The key observation we use here is that if \(\lambda\) is an eigenvalue of \(A\) of algebraic multiplicity \(m\text{,}\) then we can find certain \(m\) linearly independent vectors solving the equation \({(A-\lambda I)}^m \vec{v} = \vec{0}\text{.}\) We will call these *generalized eigenvectors*.

Let us continue with the example \(A = \left[ \begin{smallmatrix}
3 & 1 \\ 0 & 3
\end{smallmatrix} \right]\) and the equation \({\vec{x}\,}' = A\vec{x}\text{.}\) We have an eigenvalue \(\lambda=3\) of (algebraic) multiplicity 2 and defect 1. We have found one eigenvector \(\vec{v}_1 = \left[ \begin{smallmatrix} 1 \\ 0 \end{smallmatrix} \right]\text{.}\) We have the solution

\begin{equation*}
\vec{x}_1 = \vec{v}_1 e^{3t} .
\end{equation*}

In this case, let us try (in the spirit of repeated roots of the characteristic equation for a single equation) another solution of the form

\begin{equation*}
\vec{x}_2 = ( \vec{v}_2 + \vec{v}_1 t )\, e^{3t} .
\end{equation*}

We differentiate to get

\begin{equation*}
{\vec{x}_2{}}' =
\vec{v}_1 e^{3t} +
3 ( \vec{v}_2 + \vec{v}_1 t )\, e^{3t}
=
( 3 \vec{v}_2 + \vec{v}_1 )\, e^{3t} + 3 \vec{v}_1 t e^{3t} .
\end{equation*}

As we are assuming that \(\vec{x}_2\) is a solution, \({\vec{x}_2{}}'\) must equal \(A \vec{x}_2\text{,}\) and

\begin{equation*}
A \vec{x}_2 =
A ( \vec{v}_2 + \vec{v}_1 t )\, e^{3t}
=
A \vec{v}_2 e^{3t} + A \vec{v}_1 t e^{3t} .
\end{equation*}

By looking at the coefficients of \(e^{3t}\) and \(t e^{3t}\) we see \(3 \vec{v}_2 + \vec{v}_1 = A \vec{v}_2\) and \(3 \vec{v}_1 = A \vec{v}_1\text{.}\) This means that

\begin{equation*}
(A-3I)\vec{v}_2 = \vec{v}_1,
\qquad \text{and} \qquad
(A-3I)\vec{v}_1 = \vec{0}.
\end{equation*}

Therefore, \(\vec{x}_2\) is a solution if these two equations are satisfied. The second equation is satisfied as \(\vec{v}_1\) is an eigenvector. So, if we can find a \(\vec{v}_2\) that solves \((A-3I)\vec{v}_2 = \vec{v}_1\text{,}\) then we are done. This is just a bunch of linear equations to solve and we are by now very good at that. Let us solve \((A-3I)\vec{v}_2 = \vec{v}_1\text{.}\) Write

\begin{equation*}
\begin{bmatrix}
0 & 1 \\ 0 & 0
\end{bmatrix}
\begin{bmatrix}
a \\ b
\end{bmatrix}
=
\begin{bmatrix}
1 \\ 0
\end{bmatrix} .
\end{equation*}

By inspection we see that letting \(a=0\) (\(a\) could be anything in fact) and \(b=1\) does the job. Hence we can take \(\vec{v}_2 =
\left[ \begin{smallmatrix} 0 \\ 1 \end{smallmatrix} \right]\text{.}\) Our general solution to \({\vec{x}\,}' = A\vec{x}\) is

\begin{equation*}
\vec{x} =
c_1
\begin{bmatrix}
1 \\ 0
\end{bmatrix}
e^{3t}
+
c_2
\left(
\begin{bmatrix}
0 \\ 1
\end{bmatrix}
+
\begin{bmatrix}
1 \\ 0
\end{bmatrix}
t
\right)
\,
e^{3t}
=
\begin{bmatrix}
c_1 e^{3t}+c_2 te^{3t} \\
c_2 e^{3t}
\end{bmatrix} .
\end{equation*}

Let us check that we really do have the solution. First \(x_1' =
c_1 3 e^{3t}+c_2 e^{3t} + 3 c_2 te^{3t} = 3 x_1 + x_2\text{.}\) Good. Now \(x_2' = 3 c_2 e^{3t} = 3x_2\text{.}\) Good.

In the example, if we plug \((A-3I)\vec{v}_2 = \vec{v}_1\) into \((A-3I)\vec{v}_1 = \vec{0}\) we find

\begin{equation*}
(A-3I)(A-3I) \vec{v}_2 = \vec{0},
\qquad \text{or} \qquad
{(A-3I)}^2\vec{v}_2 = \vec{0}.
\end{equation*}

Furthermore, if \((A-3I) \vec{w} \not= \vec{0}\text{,}\) then \((A-3I) \vec{w}\) is an eigenvector, a multiple of \(\vec{v}_1\text{.}\) In this \(2 \times 2\) case \({(A-3I)}^2\) is just the zero matrix (exercise). So any vector \(\vec{w}\) solves \({(A-3I)}^2\vec{w} = \vec{0}\) and we just need a \(\vec{w}\) such that \((A-3I)\vec{w} \not= \vec{0}\text{.}\) Then we could use \(\vec{w}\) for \(\vec{v}_2\text{,}\) and \((A-3I)\vec{w}\) for \(\vec{v}_1\text{.}\)

Note that the system \({\vec{x}\,}' = A \vec{x}\) has a simpler solution since \(A\) is a so-called *upper triangular matrix*, that is every entry below the diagonal is zero. In particular, the equation for \(x_2\) does not depend on \(x_1\text{.}\) Mind you, not every defective matrix is triangular.

######
Exercise 3.7.1

Solve \({\vec{x}\,}' = \left[ \begin{smallmatrix}
3 & 1 \\ 0 & 3
\end{smallmatrix} \right] \vec{x}\) by first solving for \(x_2\) and then for \(x_1\) independently. Check that you got the same solution as we did above.

Let us describe the general algorithm. Suppose that \(\lambda\) is an eigenvalue of multiplicity 2, defect 1. First find an eigenvector \(\vec{v}_1\) of \(\lambda\text{.}\) That is, \(\vec{v}_1\) solves \((A-\lambda I)\vec{v}_1 = \vec{0}\text{.}\) Then, find a vector \(\vec{v}_2\) such that

\begin{equation*}
(A-\lambda I)\vec{v}_2 = \vec{v}_1 .
\end{equation*}

This gives us two linearly independent solutions

\begin{equation*}
\begin{aligned}
\vec{x}_1 & = \vec{v}_1 e^{\lambda t} , \\
\vec{x}_2 & = \left( \vec{v}_2 + \vec{v}_1 t \right) e^{\lambda t} .
\end{aligned}
\end{equation*}

This machinery can also be generalized to higher multiplicities and higher defects. We will not go over this method in detail, but let us just sketch the ideas. Suppose that \(A\) has an eigenvalue \(\lambda\) of multiplicity \(m\text{.}\) We find vectors such that

\begin{equation*}
{(A - \lambda I)}^k \vec{v} = \vec{0},
\qquad \text{but} \qquad
{(A - \lambda I)}^{k-1} \vec{v} \not= \vec{0}.
\end{equation*}

Such vectors are called *generalized eigenvectors* (then \(\vec{v}_1 = {(A - \lambda I)}^{k-1} \vec{v}\) is an eigenvector). For every eigenvector \(\vec{v}_1\) we find a chain of generalized eigenvectors \(\vec{v}_2\) through \(\vec{v}_k\) such that:

\begin{equation*}
\begin{aligned}
(A - \lambda I) \vec{v}_1 & = \vec{0} , \\
(A - \lambda I) \vec{v}_2 & = \vec{v}_1 , \\
& ~~\vdots \\
(A - \lambda I) \vec{v}_k & = \vec{v}_{k-1} .
\end{aligned}
\end{equation*}

We form the linearly independent solutions

\begin{equation*}
\begin{aligned}
\vec{x}_1 & = \vec{v}_1 e^{\lambda t} , \\
\vec{x}_2 & = ( \vec{v}_2 + \vec{v}_1 t ) \, e^{\lambda t} , \\
& ~~\vdots \\
\vec{x}_k & = \left( \vec{v}_k + \vec{v}_{k-1} t +
\vec{v}_{k-2} \frac{t^2}{2} +
\cdots + \vec{v}_2 \frac{t^{k-2}}{(k-2)!} + \vec{v}_1 \frac{t^{k-1}}{(k-1)!}
\right) \, e^{\lambda t} .
\end{aligned}
\end{equation*}

Recall that \(k! = 1 \cdot 2 \cdot 3 \cdots (k-1) \cdot k\) is the factorial. We proceed to find chains until we form \(m\) linearly independent solutions (\(m\) is the multiplicity). You may need to find several chains for every eigenvalue.

###
Subsection 3.7.3 Exercises

######
Exercise 3.7.2

Let \(A = \left[ \begin{smallmatrix} 5 & -3 \\ 3 & -1 \end{smallmatrix} \right]\text{.}\) Find the general solution of \({\vec{x}\,}' = A \vec{x}\text{.}\)

######
Exercise 3.7.3

Let \(A = \left[ \begin{smallmatrix}
5 & -4 & 4 \\
0 & 3 & 0 \\
-2 & 4 & -1
\end{smallmatrix} \right]\text{.}\) a) What are the eigenvalues? b) What is/are the defect(s) of the eigenvalue(s)? c) Find the general solution of \({\vec{x}\,}' = A \vec{x}\text{.}\)

######
Exercise 3.7.4

Let \(A = \left[ \begin{smallmatrix} 2 & 1 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 2 \end{smallmatrix} \right]\text{.}\) a) What are the eigenvalues? b) What is/are the defect(s) of the eigenvalue(s)? c) Find the general solution of \({\vec{x}\,}' = A \vec{x}\) in two different ways and verify you get the same answer.

######
Exercise 3.7.5

Let \(A = \left[ \begin{smallmatrix}
0 & 1 & 2 \\
-1 & -2 & -2 \\
-4 & 4 & 7
\end{smallmatrix} \right]\text{.}\) a) What are the eigenvalues? b) What is/are the defect(s) of the eigenvalue(s)? c) Find the general solution of \({\vec{x}\,}' = A \vec{x}\text{.}\)

######
Exercise 3.7.6

Let \(A = \left[ \begin{smallmatrix}
0 & 4 & -2 \\
-1 & -4 & 1 \\
0 & 0 & -2
\end{smallmatrix} \right]\text{.}\) a) What are the eigenvalues? b) What is/are the defect(s) of the eigenvalue(s)? c) Find the general solution of \({\vec{x}\,}' = A \vec{x}\text{.}\)

######
Exercise 3.7.7

Let \(A = \left[ \begin{smallmatrix}
2 & 1 & -1 \\
-1 & 0 & 2 \\
-1 & -2 & 4
\end{smallmatrix} \right]\text{.}\) a) What are the eigenvalues? b) What is/are the defect(s) of the eigenvalue(s)? c) Find the general solution of \({\vec{x}\,}' = A \vec{x}\text{.}\)

######
Exercise 3.7.8

Suppose that \(A\) is a \(2 \times 2\) matrix with a repeated eigenvalue \(\lambda\text{.}\) Suppose that there are two linearly independent eigenvectors. Show that \(A = \lambda I\text{.}\)

######
Exercise 3.7.101

Let \(A =
\left[ \begin{smallmatrix}
1 & 1 & 1 \\
1 & 1 & 1 \\
1 & 1 & 1
\end{smallmatrix}\right]\text{.}\) a) What are the eigenvalues? b) What is/are the defect(s) of the eigenvalue(s)? c) Find the general solution of \(\vec{x}\,' = A\vec{x}\text{.}\)

Answera) \(3,0,0\) b) No defects. c) \(\vec{x} =
C_1
\left[ \begin{smallmatrix}
1 \\ 1 \\ 1
\end{smallmatrix}\right]
e^{3t}
+
C_2
\left[ \begin{smallmatrix}
1 \\ 0 \\ -1
\end{smallmatrix}\right]
+
C_3
\left[ \begin{smallmatrix}
0 \\ 1 \\ -1
\end{smallmatrix}\right]\)

######
Exercise 3.7.102

Let \(A =
\left[ \begin{smallmatrix}
1 & 3 & 3 \\
1 & 1 & 0 \\
-1 & 1 & 2 \\
\end{smallmatrix}\right]\text{.}\) a) What are the eigenvalues? b) What is/are the defect(s) of the eigenvalue(s)? c) Find the general solution of \(\vec{x}\,' = A\vec{x}\text{.}\)

Answera) \(1,1,2\)

b) Eigenvalue 1 has a defect of 1

c) \(\vec{x} =
C_1
\left[ \begin{smallmatrix}
0 \\ 1 \\ -1
\end{smallmatrix}\right]
e^{t}
+
C_2
\left(
\left[ \begin{smallmatrix}
1 \\ 0 \\ 0
\end{smallmatrix}\right]
+
t
\left[ \begin{smallmatrix}
0 \\ 1 \\ -1
\end{smallmatrix}\right]
\right)
e^{t}
+
C_3
\left[ \begin{smallmatrix}
3 \\ 3 \\ -2
\end{smallmatrix}\right]
e^{2t}\)

######
Exercise 3.7.103

Let \(A =
\left[ \begin{smallmatrix}
2 & 0 & 0 \\
-1 & -1 & 9 \\
0 & -1 & 5
\end{smallmatrix}\right]\text{.}\) a) What are the eigenvalues? b) What is/are the defect(s) of the eigenvalue(s)? c) Find the general solution of \(\vec{x}\,' = A\vec{x}\text{.}\)

Answera) \(2,2,2\)

b) Eigenvalue 2 has a defect of 2

c) \(\vec{x} =
C_1
\left[ \begin{smallmatrix}
0 \\ 3 \\ 1
\end{smallmatrix}\right]
e^{2t}
+
C_2
\left(
\left[ \begin{smallmatrix}
0 \\ -1 \\ 0
\end{smallmatrix}\right]
+
t
\left[ \begin{smallmatrix}
0 \\ 3 \\ 1
\end{smallmatrix}\right]
\right)
e^{2t}
+
C_3
\left(
\left[ \begin{smallmatrix}
1 \\ 0 \\ 0
\end{smallmatrix}\right]
+
t
\left[ \begin{smallmatrix}
0 \\ -1 \\ 0
\end{smallmatrix}\right]
+
\frac{t^2}{2}
\left[ \begin{smallmatrix}
0 \\ 3 \\ 1
\end{smallmatrix}\right]
\right)
e^{2t}\)

######
Exercise 3.7.104

Let \(A =
\left[ \begin{smallmatrix}
a & a \\
b & c
\end{smallmatrix}\right]\text{,}\) where \(a\text{,}\) \(b\text{,}\) and \(c\) are unknowns. Suppose that \(5\) is a doubled eigenvalue of defect 1, and suppose that \(\left[ \begin{smallmatrix}
1 \\ 0
\end{smallmatrix}\right]\) is a corresponding eigenvector. Find \(A\) and show that there is only one such matrix \(A\text{.}\)

Answer\(A = \left[ \begin{smallmatrix}
5 & 5 \\ 0 & 5
\end{smallmatrix}\right]\)