You have surely heard the story about the flap of a butterfly wing in the Amazon causing hurricanes in the North Atlantic. In a prior section, we mentioned that a small change in initial conditions of the planets can lead to very different configuration of the planets in the long term. These are examples of chaotic systems. Mathematical chaos is not really chaos, there is precise order behind the scenes. Everything is still deterministic. However a chaotic system is extremely sensitive to initial conditions. This also means even small errors induced via numerical approximation create large errors very quickly, so it is almost impossible to numerically approximate for long times. This is large part of the trouble as chaotic systems cannot be in general solved analytically.

Take the weather for example. As a small change in the initial conditions (the temperature at every point of the atmosphere for example) produces drastically different predictions in relatively short time, we cannot accurately predict weather. This is because we do not actually know the exact initial conditions, we measure temperatures at a few points with some error and then we somehow estimate what is in between. There is no way we can accurately measure the effects of every butterfly wing. Then we solve numerically introducing new errors. That is why you should not trust weather prediction more than a few days out.

The idea of chaotic behavior was first noticed by Edward Lorenz^{ 1 }Edward Norton Lorenz (1917–2008) was an American mathematician and meteorologist. in the 1960s when trying to model thermally induced air convection (movement). The equations Lorentz was looking at form the relatively simple looking system:

A small change in the initial conditions yield a very different solution after a reasonably short time.

A very simple example the reader can experiment with, which displays chaotic behavior, is a double pendulum. The equations that govern this system are somewhat complicated and their derivation is quite tedious, so we will not bother to write them down. The idea is to put a pendulum on the end of another pendulum. If you look at the movement of the bottom mass, the movement will appear chaotic. This type of system is a basis for a whole number of office novelty desk toys. It is very simple to build a version. Take a piece of a string, and tie two heavy nuts at different points of the string; one at the end, and one a bit above. Now give the bottom nut a little push, as long as the swings are not too big and the string stays tight, you have a double pendulum system.

Subsection8.5.1Duffing equation and strange attractors

Let us study the so-called Duffing equation:

\begin{equation*}
x'' + a x' + bx + cx^3 = C \cos(\omega t) .
\end{equation*}

Here \(a\text{,}\) \(b\text{,}\) \(c\text{,}\) \(C\text{,}\) and \(\omega\) are constants. You will recognize that except for the \(c x^3\) term, this equation looks like a forced mass-spring system. The \(c x^3\) term comes up when the spring does not exactly obey Hooke's law (which no real-world spring actually does obey exactly). When \(c\) is not zero, the equation does not have a nice closed form solution, so we have to resort to numerical solutions as is usual for nonlinear systems. Not all choices of constants and initial conditions exhibit chaotic behavior. Let us study

The equation is not autonomous, so we cannot draw the vector field in the phase plane. We can still draw the trajectories.

In Figure 9.5.1 we plot trajectories for \(t\) going from 0 to 15, for two very close initial conditions \((2,3)\) and \((2,2.9)\text{,}\) and also the solutions in the \((x,t)\) space. The two trajectories are close at first, but after a while diverge significantly. This sensitivity to initial conditions is precisely what we mean by the system behaving chaotically.

Let us see the long term behavior. In Figure 9.5.2, we plot the behavior of the system for initial conditions \((2,3)\text{,}\) but for much longer period of time. Note that for this period of time it was necessary to use a ridiculously large number of steps in the numerical algorithm used to produce the graph, as even small errors quickly propagate^{ 2 }In fact for reference, 30,000 steps were used with the Runge-Kutta algorithm, see exercises in Section 1.7.. From the graph it is hard to see any particular pattern in the shape of the solution except that it seems to oscillate, but each oscillation appears quite unique. The oscillation is expected due to the forcing term.

In general it is very difficult to analyze chaotic systems, or to find the order behind the madness, but let us try to do something that we did for the standard mass-spring system. One way we analyzed the system is that we figured out what was the long term behavior (not dependent on initial conditions). From the figure above it is clear that we will not get a nice description of the long term behavior for this chaotic system, but perhaps we can figure out some order to what happens on each “oscillation” and what do these oscillations have in common.

The concept we explore is that of a Poincarè section^{ 3 }Named for the French polymath Jules Henri Poincarè (1854–1912).. Instead of looking at \(t\) in a certain interval, we look at where the system is at a certain sequence of points in time. Imagine flashing a strobe at a certain fixed frequency and drawing the points where the solution is during the flashes. The right strobing frequency depends on the system in question. The correct frequency to use for the forced Duffing equation (and other similar systems) is the frequency of the forcing term. For the Duffing equation above, find a solution \(\bigl(x(t),y(t)\bigr)\text{,}\) and look at the points

As we are really not interested in the transient part of the solution, that is, the part of the solution that depends on the initial condition we skip some number of steps in the beginning. For example, we might skip the first 100 such steps and start plotting points at \(t = 100(2\pi)\text{,}\) that is

The plot of these points is the Poincarè section. After plotting enough points, a curious pattern emerges in Figure 9.5.3 (the left hand picture), a so-called strange attractor.

If we have a sequence of points, then an attractor is a set towards which the points in the sequence eventually get closer and closer to, that is, they are attracted. The Poincarè section is not really the attractor itself, but as the points are very close to it, we see its shape. The strange attractor in the figure is a very complicated set. In fact, it has fractal structure, that is, if you zoom in as far as you want, you keep seeing the same complicated structure.

The initial condition makes no difference. If we start with a different initial condition, the points eventually gravitate towards the attractor, and so as long as we throw away the first few points, we get the same picture. Similarly small errors in the numerical approximations do not matter here.

An amazing thing is that a chaotic system such as the Duffing equation is not random at all. There is a very complicated order to it, and the strange attractor says something about this order. We cannot quite say what state the system will be in eventually, but given a fixed strobing frequency we can narrow it down to the points on the attractor.

If we use a phase shift, for example \(\nicefrac{\pi}{4}\text{,}\) and look at the times

we obtain a slightly different looking attractor. The picture is the right hand side of Figure 9.5.3. It is as if we had rotated, distorted slightly, and then moved the original. Therefore for each phase shift you can find the set of points towards which the system periodically keeps coming back to.

You should study the pictures and notice especially the scales—where are these attractors located in the phase plane. Notice the regions where the strange attractor lives and compare it to the plot of the trajectories in Figure 9.5.1.

Let us compare the discussion in this section to the discussion in Section 2.6 about forced oscillations. Take the equation

This is like the Duffing equation, but with no \(x^3\) term. The steady periodic solution is of the form

\begin{equation*}
x = C \cos (\omega t + \gamma) .
\end{equation*}

Strobing using the frequency \(\omega\) we would obtain a single point in the phase space. So the attractor in this setting is a single point—an expected result as the system is not chaotic. In fact it was the opposite of chaotic. Any difference induced by the initial conditions dies away very quickly, and we settle into always the same steady periodic motion.

Subsection8.5.2The Lorenz system

In two dimensions to have the kind of chaotic behavior we are looking for, we have to study forced, or non-autonomous, systems such as the Duffing equation. Due to the Poincarè-Bendixson Theorem, if an autonomous two-dimensional system has a solution that exists for all time in the future and does not go towards infinity, then it is periodic or it tends towards a periodic solution. Hardly the chaotic behavior we are looking for.

The Lorenz system is an autonomous system in three dimensions exhibiting chaotic behavior. See the Figure 9.5.4 for a sample trajectory.

The solutions tend to an attractor in space, the so-called Lorenz attractor. In this case no strobing is necessary. Again we cannot quite see the attractor itself, but if we try to follow a solution for long enough, as in the figure, we will get a pretty good picture of what the attractor looks like.

The path is not just a repeating figure-eight. The trajectory will spin some seemingly random number of times on the left, then spin a number of times on the right, and so on. As this system arose in weather prediction, one can perhaps imagine a few days of warm weather and then a few days of cold weather, where it is not easy to predict when the weather will change, just as it is not really easy to predict far in advance when the solution will jump onto the other side. See Figure 9.5.5 for a plot of the \(x\) component of the solution drawn above.

Subsection8.5.3Exercises

Exercise8.5.1

For the non-chaotic equation \(x''+2p x' + \omega_0^2 x = \frac{F_0}{m} \cos (\omega t)\text{,}\) suppose we strobe with frequency \(\omega\) as we mentioned above. Use the known steady periodic solution to find precisely the point which is the attractor for the Poincarè section.

Exercise8.5.2

(project) A simple fractal attractor can be drawn via the following chaos game. Draw three points of a triangle (just the vertices) and number them, say \(p_1\text{,}\) \(p_2\) and \(p_3\text{.}\) Start with some random point \(p\) (does not have to be one of the three points above) and draw it. Roll a die, and use it to pick of the \(p_1\text{,}\) \(p_2\text{,}\) or \(p_3\) randomly (for example 1 and 4 mean \(p_1\text{,}\) 2 and 5 mean \(p_2\text{,}\) and 3 and 6 mean \(p_3\)). Suppose we picked \(p_2\text{,}\) then let \(p_{\text{new}}\) be the point exactly halfway between \(p\) and \(p_2\text{.}\) Draw this point and let \(p\) now refer to this new point \(p_{\text{new}}\text{.}\) Rinse, repeat. Try to be precise and draw as many iterations as possible. Your points should be attracted to the so-called Sierpinski triangle. A computer was used to run the game for 10,000 iterations to obtain the picture in Figure 9.5.8.

Exercise8.5.3

(project) Construct the double pendulum described in the text with a string and two nuts (or heavy beads). Play around with the position of the middle nut, and perhaps use different weight nuts. Describe what you find.

Exercise8.5.4

(computer project) Use a computer software (such as Matlab, Octave, or perhaps even a spreadsheet), plot the solution of the given forced Duffing equation with Euler's method. Plotting the solution for \(t\) from 0 to 100 with several different (small) step sizes. Discuss.

Exercise8.5.101

Find critical points of the Lorenz system and the associated linearizations.