[Go to the Notes on Diffy Qs home page] [PDF version]
[next] [prev] [prev-tail] [tail] [up]
Note: 1 lecture (or less), §1.2 in [EP]
A first order ODE is an equation of the form
![]() |
or just
![]() |
In general, there is no simple formula or procedure one can follow to find solutions. In the next few
lectures we will look at special cases where solutions are not difficult to obtain. In this section, let us
assume that
is a function of
alone, that is, the equation is
![]() | (1.1) |
We could just integrate (antidifferentiate) both sides with respect to
.
![]() |
that is
![]() |
This
is actually the general solution. So to solve (1.1), we find some antiderivative of
and then
we add an arbitrary constant to get the general solution.
Now is a good time to discuss a point about calculus notation and terminology. Calculus
textbooks muddy the waters by talking about integral as primarily the so-called indefinite integral.
The indefinite integral is really the antiderivative (in fact the whole one-parameter family of
antiderivatives). There really exists only one integral and that is the definite integral. The only reason for
the indefinite integral notation is that we can always write an antiderivative as a (definite)
integral. That is, by the fundamental theorem of calculus we can always write
as
![]() |
Hence the terminology to integrate when we may really mean to antidifferentiate. Integration is just one way to compute the antiderivative (and it is a way that always works, see the following examples). Integration is defined as the area under the graph, it only happens to also compute antiderivatives. For sake of consistency, we will keep using the indefinite integral notation when we want an antiderivative, and you should always think of the definite integral.
Example 1.1.1: Find the general solution of
.
Elementary calculus tells us that the general solution must be
. Let us check:
. We have gotten precisely our equation back.
Normally, we also have an initial condition such as
for some two numbers
and
(
is usually 0, but not always). We can then write the solution as a definite integral in a nice way. Suppose
our problem is
,
. Then the solution is
![]() | (1.2) |
Let us check! We compute
(by fundamental theorem of calculus) and by Jupiter,
is a
solution. Is it the one satisfying the initial condition? Well,
. It
is!
Do note that the definite integral and the indefinite integral (antidifferentiation) are completely different beasts. The definite integral always evaluates to a number. Therefore, (1.2) is a formula we can plug into the calculator or a computer, and it will be happy to calculate specific values for us. We will easily be able to plot the solution and work with it just like with any other function. It is not so crucial to always find a closed form for the antiderivative.
![]() |
By the preceding discussion, the solution must be
![]() |
Here is a good way to make fun of your friends taking second semester calculus. Tell them to find the closed form solution. Ha ha ha (bad math joke). It is not possible (in closed form). There is absolutely nothing wrong with writing the solution as a definite integral. This particular integral is in fact very important in statistics.
Using this method, we can also solve equations of the form
![]() |
Let us write the equation in Leibniz notation.
![]() |
Now we use the inverse function theorem from calculus to switch the roles of
and
to
obtain
![]() |
What we are doing seems like algebra with
and
. It is tempting to just do algebra with
and
as if they were numbers. And in this case it does work. Be careful, however, as this sort of hand-waving
calculation can lead to trouble, especially when more than one independent variable is involved. At this
point we can simply integrate
![]() |
Finally, we try to solve for
.
Example 1.1.3: Previously, we guessed
(for some
) has solution
. We can
actually do it now. First we note that
is a solution. Henceforth, we assume
. We
write
![]() |
We integrate to obtain
![]() |
where
is an arbitrary constant. Now we solve for
(actually for
).
![]() |
If we replace
with an arbitrary constant
we can get rid of the absolute value bars (we can do this
as
was arbitrary). In this way, we also incorporate the solution
. We get the same general
solution as we guessed before,
.
Example 1.1.4: Find the general solution of
.
First we note that
is a solution. We can now assume that
. Write
![]() |
We integrate to get
![]() |
We solve for
. So the general solution is
![]() |
Note the singularities of the solution. If for example
, then the solution “blows up” as we approach
. Generally, it is hard to tell from just looking at the equation itself how the solution is going to
behave. The equation
is very nice and defined everywhere, but the solution is only defined on
some interval
or
.
Classical problems leading to differential equations solvable by integration are problems dealing with velocity, acceleration and distance. You have surely seen these problems before in your calculus class.
Example 1.1.5: Suppose a car drives at a speed
meters per second, where
is time in
seconds. How far did the car get in 2 seconds (starting at
)? How far in 10 seconds?
Let
denote the distance the car traveled. The equation is
![]() |
We can just integrate this equation to get that
![]() |
We still need to figure out
. We know that when
, then
. That is,
.
So
![]() |
Thus
and
![]() |
Now we just plug in to get where the car is at 2 and at 10 seconds. We obtain
![]() |
Example 1.1.6: Suppose that the car accelerates at a rate of
. At time
the car is at the
1 meter mark and is traveling at 10 m/s. Where is the car at time
.
Well this is actually a second order problem. If
is the distance traveled, then
is the velocity, and
is the acceleration. The equation with initial conditions is
![]() |
Well, what if we call
, and then we have the problem
![]() |
Once we solve for
, we can then integrate and find
.
Exercise 1.1.9: A spaceship is traveling at the speed
km/s (
is time in seconds). It is
pointing directly away from earth and at time
it is 1000 kilometers from earth. How far from
earth is it at one minute from time
?
Note: 1 lecture, §1.3 in [EP]
At this point it may be good to first try the Lab I and/or Project I from the IODE website: http://www.math.uiuc.edu/iode/.
As we said, the general first order equation we are studying looks like
![]() |
In general, we cannot simply solve these kinds of equations explicitly. It would be good if we could at least figure out the shape and behavior of the solutions, or if we could even find approximate solutions for any equation.
As you have seen in IODE Lab I (if you did it), the equation
gives you a slope at each point
in the
-plane. We can plot the slope at lots of points as a short line through the point
with the
slope
. See Figure 1.1.
.
with
a graph of solutions satisfying
,
, and
.
We call this picture the slope field of the equation. If we are given a specific initial condition
, we can look at the location
and follow the slopes. See Figure 1.2.
By looking at the slope field we can get a lot of information about the behavior of solutions. For
example, in Figure 1.2 we can see what the solutions do when the initial conditions are
,
and
. Note that a small change in the initial condition causes quite different behavior.
On the other hand, plotting a few solutions of the equation
, we see that no matter what
is,
all solutions tend to zero as
tends to infinity. See Figure 1.3.
We wish to ask two fundamental questions about the problem
![]() |
What do you think is the answer? The answer seems to be yes to both does it not? Well, pretty much. But there are cases when the answer to either question can be no.
Since generally the equations we encounter in applications come from real life situations, it seems logical that a solution always exists. It also has to be unique if we believe our universe is deterministic. If the solution does not exist, or if it is not unique, we have probably not devised the correct model. Hence, it is good to know when things go wrong and why.
Example 1.2.1: Attempt to solve:
![]() |
Integrate to find the general solution
. Note that the solution does not exist at
.
See Figure 1.4.
It is actually hard to tell from the slope field that the solution will not be unique. Is there any
hope? Of course there is. It turns out that the following theorem is true. It is known as Picard’s
theorem
.
Theorem 1.2.1 (Picard’s theorem on existence and uniqueness). If
is continuous (as a
function of two variables) and
exists and is continuous near some
, then a solution
to
![]() |
exists (at least for some small interval of
’s) and is unique.
Note that the problems
,
and
,
do not satisfy the hypothesis of
the theorem. Even if we can use the theorem, we ought to be careful about this existence business. It is
quite possible that the solution only exists for a short while.
Example 1.2.3: For some constant
, solve:
![]() |
We know how to solve this equation. First assume that
, so
is not equal to zero at least
for some
near 0. So
, so
, so
. If
, then
so
![]() |
If
, then
is a solution.
For example, when
the solution “blows up” at
. Hence, the solution does not
exist for all
even if the equation is nice everywhere. The equation
certainly looks
nice.
For the most of this course we will be interested in equations where existence and uniqueness holds,
and in fact holds “globally” unlike for the equation
.
Exercise 1.2.1: Sketch direction field for
. How do the solutions behave as
grows?
Can you guess a particular solution by looking at the direction field?
Note: 1 lecture, §1.4 in [EP]
When a differential equation is of the form
, we can just integrate:
.
Unfortunately this method no longer works for the general form of the equation
. Integrating
both sides yields
![]() |
Notice dependence on
in the integral.
Let us suppose that the equation is separable. That is, let us consider
![]() |
for some functions
and
. Let us write the equation in the Leibniz notation
![]() |
Then we rewrite the equation as
![]() |
Now both sides look like something we can integrate. We obtain
![]() |
If we can find closed form expressions for these two integrals, we can, perhaps, solve for
.
Example 1.3.1: Take the equation
![]() |
First note that
is a solution, so assume
from now on. Write the equation as
,
then
![]() |
We compute the antiderivatives to get
![]() |
Or
![]() |
where
is some constant. Because
is a solution and because of the absolute value we
actually can write:
![]() |
for any number
(including zero or negative).
We check:
![]() |
Yay!
We should be a little bit more careful with this method. You may be worried that we were integrating in two different variables. We seemed to be doing a different operation to each side. Let us work this method out more rigorously.
![]() |
We rewrite the equation as follows. Note that
is a function of
and so is
!
![]() |
We integrate both sides with respect to
.
![]() |
We can use the change of variables formula.
![]() |
And we are done.
It is clear that we might sometimes get stuck even if we can do the integration. For example, take the separable equation
![]() |
We separate variables
![]() |
Now we integrate to get
![]() |
Or maybe the easier looking expression (where
):
![]() |
It is not easy to find the solution explicitly as it is hard to solve for
. We will, therefore, call this solution
an implicit solution. It is easy to check that implicit solutions still satisfy the differential equation. In this
case, we differentiate to get
![]() |
It is simple to see that the differential equation holds. If you want to compute values for
you might have
to be tricky. For example, you can graph
as a function of
, and then flip your paper. Computers are
also good at some of these tricks, but you have to be careful.
We note above that the equation also has a solution
. In this case, it turns out that the general
solution is
together with
. These outlying solutions such as
are
sometimes called singular solutions.
First factor the right hand side to obtain
![]() |
We separate variables, integrate and solve for

to get
(or
, etc…). The solution we are
seeking is, therefore,
![]() |
Example 1.3.3: Suppose Bob made a cup of coffee, and the water was boiling (100 degrees
Celsius) at time
minutes. Suppose Bob likes to drink his coffee at 70 degrees. Let the ambient
(room) temperature be 26 degrees. Furthermore, suppose Bob measured the temperature of the
coffee at 1 minute and found that it dropped to 95 degrees. When should Bob start drinking?
Let
be the temperature of coffee, let
be the ambient (room) temperature. Then for some
the
temperature of coffee is:
![]() |
For our setup
,
,
. We separate variables and integrate (
and
will
denote arbitrary constants)

. We plug in the first condition
and hence
. Now we
have
. We plug in
. Solving for
we get
.
Now we solve for the time
that gives us a temperature of 70 degrees. That is, we solve
to get
minutes. So Bob can begin to drink the coffee at about 7
and a half minutes from the time Bob made it. Probably about the amount of time it took us to calculate
how long it would take.
Example 1.3.4: Find the general solution to
(including singular solutions).
First note that
is a solution (a singular solution). So assume that
and write

Exercise 1.3.11: Find explicit solution for
,
. It is alright to leave a definite
integral in your answer.
Exercise 1.3.12: Suppose a cup of coffee is at 100 degrees Celsius at time
, it is at 70 degrees
at
minutes, and it is at 50 degrees at
minutes. Compute the ambient temperature.
Note: 1 lecture, §1.5 in [EP]
One of the most important types of equations we will learn how to solve are the so-called linear equations. In fact, the majority of the course will focus on linear equations. In this lecture we will focus on the first order linear equation. A first order equation is linear if we can put it into the following form:
![]() | (1.3) |
The word “linear” here means linear in
. The dependence on
can be more complicated.
Solutions of linear equations have nice properties. For example, the solution exists wherever
and
are defined, and has the same regularity (read: it is just as nice). But most importantly for us right
now, there is a method for solving linear first order equations.
First we find a function
such that
![]() |
Then we can multiply both sides of (1.3) by
to obtain
![]() |
Now we integrate both sides. The right hand side does not depend on
and the left hand side is written as
a derivative of a function. Afterwards, we can solve for
. The function
is called the integrating
factor and the method is called the integrating factor method.
We are looking for a function
such that if we differentiate it, we get the same function back
multiplied by
. That seems like a job for the exponential function! Let
![]() |
We compute:
![′
y + p(x)y = f(x),
∫ p(x)dx ′ ∫ p(x)dx ∫ p(x)dx
e y + e p(x)y = e f (x),
d--[ ∫ p(x)dx ] ∫ p(x)dx
dx e y = e f (x),
∫ ∫ ∫
e p(x)dxy = e p(x)dxf(x)dx + C,
(∫ )
-∫ p(x)dx ∫ p(x)dx
y = e e f (x)dx + C .](diffyqs420x.png)
Of course, to get a closed form formula for
we need to be able to find a closed form formula for the
integrals appearing above.
![]() |
First note that
and
. The integrating factor is
. We
multiply both sides of the equation by
to get
![x2 ′ x2 x-x2 x2
e y + 2xe y = e e ,
d [x2 ] x
---e y = e .
dx](diffyqs427x.png)

, so
. The solution is
![]() |
Note that we do not care which antiderivative we take when computing
. You can always add a
constant of integration, but those constants will not matter in the end.
Exercise 1.4.1: Try it! Add a constant of integration to the integral in the integrating factor and show that the solution you get in the end is the same as what we got above.
An advice: Do not try to remember the formula itself, that is way too hard. It is easier to remember the process and repeat it.
Since we cannot always evaluate the integrals in closed form, it is useful to know how to write the solution in definite integral form. A definite integral is something that you can plug into a computer or a calculator. Suppose we are given
![]() |
Look at the solution and write the integrals as definite integrals.
![]() | (1.4) |
You should be careful to properly use dummy variables here. If you now plug such a formula into a computer or a calculator, it will be happy to give you numerical answers.
Exercise 1.4.2: Check that
in formula (1.4).
Exercise 1.4.3: Write the solution of the following problem as a definite integral, but try to simplify as far as you can. You will not be able to find the solution in closed form.
![]() |
Example 1.4.2: The following is a simple application of linear equations and this type of a problem is used often in real life. For example, linear equations are used in figuring out the concentration of chemicals in bodies of water.
A 100 liter tank contains 10 kilograms of salt dissolved in 60 liters of water. Solution of water and salt (brine) with concentration of 0.1 kilograms per liter is flowing in at the rate of 5 liters a minute. The solution in the tank is well stirred and flows out at a rate of 3 liters a minute. How much salt is in the tank when the tank is full?
Let us come up with the equation. Let
denote the kilograms of salt in the tank, let
denote
the time in minutes. Then for a small change
in time, the change in
(denoted
) is
approximately
![]() |
Dividing through by
and taking the limit
we see that
![]() |
In our example, we have

![]() |
Or in the form (1.3)
![]() |
Let us solve. The integrating factor is
![]() |
We multiply both sides of the equation to get
![3∕2dx- 3∕2--3---- 3∕2
(60 + 2t) dt + (60 + 2t) 60 + 2t x = 0.5(60 + 2t) ,
d [ 3∕2 ] 3∕2
-- (60 + 2t) x = 0.5(60 + 2t) ,
dt ∫
(60 + 2t)3∕2x = 0.5(60 + 2t)3∕2dt + C,
∫ (60 + 2t)3∕2
x = (60 + 2t)-3∕2 -----------dt + C (60 + 2t)-3∕2,
2
-3∕2-1- 5∕2 -3∕2
x = (60 + 2t) 10(60 + 2t) + C (60 + 2t) ,
60 + 2t
x = -------+ C(60 + 2t)-3∕2.
10](diffyqs450x.png)
We need to find
. We know that at
,
. So
![]() |
or
![]() |
We are interested in
when the tank is full. So we note that the tank is full when
, or
when
. So
![]() |
The concentration at the end is approximately 0.1186 kg/liter and we started with 1/6 or 0.167 kg/liter.
In the exercises, feel free to leave answer as a definite integral if a closed form solution cannot be found. If you can find a closed form solution, you should give that.
Exercise 1.4.9: Suppose there are two lakes located on a stream. Clean water flows into the first lake, then the water from the first lake flows into the second lake, and then water from the second lake flows further downstream. The in and out flow from each lake is 500 liters per hour. The first lake contains 100 thousand liters of water and the second lake contains 200 thousand liters of water. A truck with 500 kg of toxic substance crashes into the first lake. Assume that the water is being continually mixed perfectly by the stream. a) Find the concentration of toxic substance as a function of time (in seconds) in both lakes. b) When will the concentration in the first lake be below 0.001 kg per liter. c) When will the concentration in the second lake be maximal.
Exercise 1.4.10: Newton’s law of cooling states that
where
is the temperature,
is time,
is the ambient temperature, and
is a constant. Suppose that
for some constants
and
. That is, the ambient temperature oscillates (for example night and
day temperatures). a) Find the general solution. b) In the long term, will the initial conditions make
much of a difference? Why or why not.
Exercise 1.4.11: Initially 5 grams of salt are dissolved in 20 liters of water. Brine with concentration of salt 2 grams of salt per liter is added at a rate of 3 liters a minute. The tank is mixed well and is drained at 3 liters a minute. How long does the process have to continue until there are 20 grams of salt in the tank?
Exercise 1.4.12: Initially a tank contains 10 liters of pure water. Brine of unknown concentration of salt is flowing in at 1 liter per minute. The water is mixed well and drained at 1 liter per minute. In 20 minutes there are 15 grams of salt in the tank. What is the concentration of salt in the incoming brine?
Note: 1 lecture, §1.6 in [EP]
Just like when solving integrals, one method to try is to change variables to end up with a simpler equation to solve.
The equation
![]() |
is neither separable nor linear. What can we do? How about trying to change variables, so that in the new
variables the equation is simpler. We will use another variable
, which we will treat as a function of
.
Let us try
![]() |
We need to figure out
in terms of
,
and
. We differentiate (in
) to obtain
. So
. We plug this into the equation to get
![]() |
In other words,
. Such an equation we know how to solve.
![]() |
So

for some constant
. Note that
and
are also solutions.
Now we need to “unsubstitute” to obtain
![]() |
and also the two solutions
or
, and
or
. We solve the first
equation for
.

gives
, but no value of
gives the solution
.
Substitution in differential equations is applied in much the same way that it is applied in calculus. You guess. Several different substitutions might work. There are some general things to look for. We summarize a few of these in a table.
| When you see | Try substituting |
| |
| |
| |
| |
| |
Usually you try to substitute in the “most complicated” part of the equation with the hopes of simplifying it. The above table is just a rule of thumb. You might have to modify your guesses. If a substitution does not work (it does not make the equation any simpler), try a different one.
There are some forms of equations where there is a general rule for
substitution that always works. One such example is the so-called Bernoulli
equation
.
![]() |
This equation looks a lot like a linear equation except for the
. If
or
, then the equation is
linear and we can solve it. Otherwise, a change of coordinates
transforms the Bernoulli equation
into a linear equation. Note that
need not be an integer.
![]() |
First, the equation is Bernoulli (
and
). We substitute
![]() |
In other words,
. So

![]() |
Now the equation is linear. We can use the integrating factor method. In particular, we will use
formula (1.4). Let us assume that
so
. This assumption is OK, as our initial
condition is
. Let us compute the integrating factor. Here
from formula (1.4) is
.

![]() |
Note that the integral in this expression is not possible to find in closed form. As we said before, it is perfectly fine to have a definite integral in our solution. Now “unsubstitute”

Another type of equations we can solve by substitution are the so-called homogeneous equations. Suppose that we can write the differential equation as
![]() |
Here we try the substitutions
![]() |
We note that the equation is transformed into
![]() |
Hence an implicit solution is
![]() |
![]() |
We put the equation into the form
. Now we substitute
to get the separable
equation
![]() |
which has a solution


, so
![]() |
Thus
and the solution we are looking for is
![]() |
Note: 1 lecture, §2.2 in [EP]
Let us consider problems of the form
![]() |
where the derivative of solutions depends only on
(the dependent variable). These types of equations
are called autonomous equations. If we think of
as time, the naming comes from the fact that the
equation is independent of time.
Let us come back to the cooling coffee problem. Newton’s law of cooling says that
![]() |
where
is the temperature,
is time,
is some constant and
is the ambient temperature. See
Figure 1.6 for an example with
and
.
Note the solution
(in the example
). We call these types of solutions the equilibrium
solutions. The points on the
axis where
are called critical points. The point
is a
critical point. In fact, each critical point corresponds to an equilibrium solution. Note also, by looking at
the graph, that the solution
is “stable” in that small perturbations in
do not lead to substantially
different solutions as
grows. If we change the initial condition a little bit, then as
we get
. We call such critical points stable. In this simple example it turns out that
all solutions in fact go to
as
. If a critical point is not stable we would say it is
unstable.
Let us consider the logistic equation
![]() |
for some positive
and
. This equation is commonly used to model population if we know the
limiting population
, that is the maximum sustainable population. The logistic equation leads to less
catastrophic predictions on world population than
. In the real world there is no such
thing as negative population, but we will still consider negative
for the purposes of the
math.
See Figure 1.7 for an example. Note two critical points,
and
. The critical point at
is stable. On the other hand the critical point at
is unstable.
It is not really necessary to find the exact solutions to talk about the long term behavior of the solutions. For example, from the above we can easily see that
![]() |
Where DNE means “does not exist.” From just looking at the slope field we cannot quite decide what
happens if
. It could be that the solution does not exist for
all the way to
. Think of the
equation
, we have seen that it only exists for some finite period of time. Same can happen here.
In our example equation above it will actually turn out that the solution does not exist for all time, but to
see that we would have to solve the equation. In any case, the solution does go to
, but it may get
there rather quickly.
Often we are interested only in the long term behavior of the solution and we would be doing
unnecessary work if we solved the equation exactly. It is easier to just look at the phase diagram or phase
portrait, which is a simple way to visualize the behavior of autonomous equations. In this case
there is one dependent variable
. So draw the
axis, mark all the critical points and then
draw arrows in between. If
draw an up arrow and if it is negative draw a down
arrow.

Armed with the phase diagram, it is easy to sketch the solutions approximately.
Exercise 1.6.1: Try sketching a few solutions simply from looking at the phase diagram. Check with the preceding graphs if you are getting the type of curves.
Once we draw the phase diagram, we can easily classify critical points as stable or unstable.

Since any mathematical model we cook up will only be an approximation to the real world, unstable points are generally bad news.
Let us think about the logistic equation with harvesting. Logistic equations are commonly used for
modeling population. Suppose an alien race really likes to eat humans. They keep a planet with humans on
it and harvest the humans at a rate of
million humans per year. Suppose
is the number of humans in
millions on the planet and
is time in years. Let
be the limiting population when no
harvesting is done.
is some constant depending on how fast humans multiply. Our equation
becomes
![]() |
Multiply out and solve for critical points
![]() |
Critical points
and
are
![]() |
Exercise 1.6.2: Draw the phase diagram for different possibilities. Note that these possibilities
are
, or
, or
and
both complex (i.e. no real solutions). Hint: Fix some simple
and
and then vary
.
For example, let
and
. When
, then
and
are distinct and positive. The
graph we will get is given in Figure 1.8. As long as the population stays above
, which
is approximately 1.55 million, then the population will not die out. If ever the population
drops below
, humans will die out, and the fast food restaurant serving them will go out of
business.
When
, then
. There is only one critical point that is unstable. When the population is
above 1.6 million it will tend towards this number. If it ever drops below 1.6 million, humans will die out
on the planet. This scenario is not one that we (as the human fast food proprietor) want to be in. A small
perturbation of the equilibrium state and we are out of business. There is no room for error. See
Figure 1.9
Finally if we are harvesting at 2 million humans per year, the population will always plummet towards zero, no matter how well stocked the planet starts. See Figure 1.10.
Exercise 1.6.3: Let
. a) Draw the phase diagram, find the critical points and mark them
stable or unstable. b) Sketch typical solutions of the equation. c) Find
for the solution
with the initial condition
.
Exercise 1.6.4: Let
. a) Draw the phase diagram for
. On this interval
mark the critical points stable or unstable. b) Sketch typical solutions of the equation. c) Find
for the solution with the initial condition
.
Exercise 1.6.5: Suppose
is positive for
, it is zero when
and
, and it
is negative for all other
. a) Draw the phase diagram for
, find the critical points and
mark them stable or unstable. b) Sketch typical solutions of the equation. c) Find
for the
solution with the initial condition
.
Exercise 1.6.6: Start with the logistic equation
. Suppose that we modify our
harvesting. That is we will only harvest an amount proportional to current population. In other
words we harvest
per unit of time for some
(Similar to earlier example with
replaced
with
). a) Construct the differential equation. b) Show that if
, then the equation is still
logistic. c) What happens when
?
Note: 1 lecture, §2.4 in [EP]
At this point it may be good to first try the Lab II and/or Project II from the IODE website: http://www.math.uiuc.edu/iode/.
As we said before, unless
is of a special form, it is generally very hard if not impossible to get
a nice formula for the solution of the problem
![]() |
What if we want to find the value of the solution at some particular
? Or perhaps we want to produce
a graph of the solution to inspect the behavior. In this section we will learn about the basics of numerical
approximation of solutions.
The simplest method for approximating a solution is Euler’s
method
.
It works as follows: We take
and compute the slope
. The slope is the change in
per
unit change in
. We follow the line for an interval of length
on the
axis. Hence if
at
,
then we will say that
(the approximate value of
at
) will be
. Rinse,
repeat! That is, compute
and
using
and
. For an example of the first two steps of the
method see Figure 1.11.
More abstractly, for any
, we compute
![]() |
The line segments we get are an approximate graph of the solution. Generally it is not exactly the solution. See Figure 1.12 for the plot of the real solution and the approximation.
with initial conditions
.
Let us see what happens with the equation
,
. Let us try to approximate
using
Euler’s method. In Figures 1.11 and 1.12 we have essentially graphically approximated
with step size 1. With step size 1 we have
. The real answer is 3. So we are
approximately 1.074 off. Let us halve the step size. Computing
with
, we find
that
, so error of about 0.791. Table 1.1 gives the values computed for various
parameters.
The difference between the actual solution and the approximate solution we will call the error. We will usually talk about just the size of the error and we do not care much about its sign. The main point is, that we usually do not know the real solution, so we only have a vague understanding of the error. If we knew the error exactly …what is the point of doing the approximation?
![]() | Approximate ![]() | Error | ![]() |
| 1 | 1.92593 | 1.07407 | |
| 0.5 | 2.20861 | 0.79139 | 0.73681 |
| 0.25 | 2.47250 | 0.52751 | 0.66656 |
| 0.125 | 2.68034 | 0.31966 | 0.60599 |
| 0.0625 | 2.82040 | 0.17960 | 0.56184 |
| 0.03125 | 2.90412 | 0.09588 | 0.53385 |
| 0.015625 | 2.95035 | 0.04965 | 0.51779 |
| 0.0078125 | 2.97472 | 0.02528 | 0.50913 |
where of
,
.We notice that except for the first few times, every time we halved the interval the error approximately halved. This halving of the error is a general feature of Euler’s method as it is a first order method. In the IODE Project II you are asked to implement a second order method. A second order method reduces the error to approximately one quarter every time we halve the interval.
Note that to get the error to be within 0.1 of the answer we had to already do 64 steps. To get it to
within 0.01 we would have to halve another three or four times, meaning doing 512 to 1024 steps. That is
quite a bit to do by hand. The improved Euler method from IODE Project II should quarter the
error every time we halve the interval, so we would have to approximately do half as many
“halvings” to get the same error. This reduction can be a big deal. With 10 halvings (starting at
) we have 1024 steps, whereas with 5 halvings we only have to do 32 steps, assuming that
the error was comparable to start with. A computer may not care about this difference for a
problem this simple, but suppose each step would take a second to compute (the function may be
substantially more difficult to compute than
). Then the difference is 32 seconds versus about
17 minutes. Note: We are not being altogether fair, a second order method would probably
double the time to do each step. Even so, it is 1 minute versus 17 minutes. Next, suppose that
we have to repeat such a calculation for different parameters a thousand times. You get the
idea.
Note that we do not know the error! How do we know what is the right step size? Essentially we keep halving the interval, and if we are lucky, we can estimate the error from a few of these calculations and the assumption that the error goes down by a factor of one half each time (if we are using standard Euler).
Exercise 1.7.2: In the table above, suppose you do not know the error. Take the approximate values of the function in the last two lines, assume that the error goes down by a factor of 2. Can you estimate the error in the last time from this? Does it (approximately) agree with the table? Now do it for the first two rows. Does this agree with the table?
Let us talk a little bit more about the example
,
. Suppose that instead of the value
we wish to find
. The results of this effort are listed in Table 1.2 for successive halvings of
.
What is going on here? Well, you should solve the equation exactly and you will notice that the
solution does not exist at
. In fact, the solution goes to infinity when you approach
.
![]() | Approximate ![]() |
| 1 | 3.16232 |
| 0.5 | 4.54329 |
| 0.25 | 6.86079 |
| 0.125 | 10.80321 |
| 0.0625 | 17.59893 |
| 0.03125 | 29.46004 |
| 0.015625 | 50.40121 |
| 0.0078125 | 87.75769 |
where of
,
.Another case when things can go bad is if the solution oscillates wildly near some point. Such an example is given in IODE Project II. In this case, the solution may exist at all points, but even a better approximation method than Euler would need an insanely small step size to compute the solution with reasonable precision. And computers might not be able to handle such a small step size anyway.
In real applications we would not use a simple method such as Euler’s. The simplest method that would probably be used in a real application is the standard Runge-Kutta method (we will not describe it here). That is a fourth order method, meaning that if we halve the interval, the error generally goes down by a factor of 16.
Choosing the right method to use and the right step size can be very tricky. There are several competing factors to consider.
is simple to
compute, we do it many times over. Large step size means faster computation, but perhaps
not the right precision.
We have seen just the beginnings of the challenges that appear in real applications. Numerical approximation of solutions to differential equations is an active research area for engineers and mathematicians. For example, the general purpose method used for the ODE solver in Matlab and Octave (as of this writing) is a method that appeared in the literature only in the 1980s.