Unit 9, lesson 55

Chaos and Sensitive Dependence

What chaos means and what it takes to establish it, with sensitive dependence on initial conditions, the Lorenz system and its volume contraction, attractors, the largest Lyapunov exponent and the forced Duffing equation.

  • Video 55
  • Sensitive Dependence and Chaos

Start the Lorenz equations at two states that differ by one part in a billion. The two computed solutions agree closely for a while and then end up in completely different places, although the equations contain no randomness at all. Is that already chaos, and what exactly has been shown? This lesson separates three kinds of statement about such a system. Some facts, such as its equilibria and the rate at which it shrinks volume, can be checked exactly, numerical solutions supply evidence about sensitive dependence, and a claim of chaos needs a definition and a proof of its own.

Key terms

Sensitive dependence on initial conditions

Definition Consider an autonomous system X˙=F(X)\dot X = F(X), where tt is time, the dot means d/dtd/dt, the state XX lies in Rn\mathbb{R}^n and FF is continuously differentiable. Each initial state then has exactly one solution, and φt(X0)\varphi_t(X_0) denotes the state at time tt of the solution with X(0)=X0X(0) = X_0. Let SS be a set of states whose solutions exist and stay in SS for all t≥0t \ge 0. The system has sensitive dependence on initial conditions on SS if there is a distance ε>0\varepsilon > 0 with the following property. For every X0X_0 in SS and every δ>0\delta > 0 there are a state Y0Y_0 in SS with ∣Y0−X0∣<δ\lvert Y_0 - X_0 \rvert < \delta and a time t≥0t \ge 0 with

∣φt(Y0)−φt(X0)∣≥ε.\lvert \varphi_t(Y_0) - \varphi_t(X_0) \rvert \ge \varepsilon.

This does not conflict with determinism, because each exact state still has exactly one future. It limits prediction instead. If small errors grow roughly like δeλt\delta e^{\lambda t} with λ>0\lambda > 0, an initial error δ\delta reaches a size Δ\Delta after a time of about

t≈1λln⁡Δδ,t \approx \frac{1}{\lambda} \ln \frac{\Delta}{\delta},

so each tenfold improvement in the initial data extends this prediction horizon by only (ln⁡10)/λ(\ln 10)/\lambda.

In plain words No matter how closely you look around a starting state, some start nearby eventually leads to a state at least a fixed distance from where the original solution is at that moment. The equation still decides the future exactly, but a measured start is never exact, so detailed prediction runs out after a finite time.

Example Instability alone already gives sensitive dependence. For x˙=x\dot x = x on S=RS = \mathbb{R} the flow is φt(x0)=x0et\varphi_t(x_0) = x_0 e^{t}, so two starts a distance dd apart are detd e^{t} apart at time tt, and every ε\varepsilon works. Yet every solution is a plain exponential, and nothing about this equation is chaotic. A stable cycle behaves the other way. In polar coordinates the system r˙=r(1−r2)\dot r = r(1 - r^2), θ˙=1\dot\theta = 1 carries every point of the unit circle around at the same angular speed, so two points on the circle keep their distance forever and there is no sensitive dependence on the circle. For the classical Lorenz system below, numerical estimates give a growth rate λ≈0.9\lambda \approx 0.9, so each extra correct digit in the start adds only about 2.52.5 time units to the horizon.

Lorenz system

Definition With tt as time and a dot for d/dtd/dt, the Lorenz system for the state X=(x,y,z)TX = (x, y, z)^{T} is

x˙=σ(y−x),y˙=x(ρ−z)−y,z˙=xy−βz,\begin{aligned} \dot x &= \sigma(y - x), \\[0.4em] \dot y &= x(\rho - z) - y, \\[0.4em] \dot z &= xy - \beta z, \end{aligned}

with constant parameters σ,ρ,β>0\sigma, \rho, \beta > 0. The classical values are σ=10\sigma = 10, ρ=28\rho = 28 and β=8/3\beta = 8/3. The right side F(X)F(X) is a polynomial, so each initial state has exactly one solution, and the only nonlinear terms are xzxz and xyxy. Replacing (x,y,z)(x, y, z) by (−x,−y,z)(-x, -y, z) carries solutions to solutions. The origin is an equilibrium for every choice of parameters. For ρ≤1\rho \le 1 it is the only one, and for ρ>1\rho > 1 there are exactly two more,

C±=(±x∗,±x∗,ρ−1),x∗=β(ρ−1).\begin{gathered} C_{\pm} = (\pm x_*, \pm x_*, \rho - 1), \\[0.4em] x_* = \sqrt{\beta(\rho - 1)}. \end{gathered}

The system also contracts volume. In general, for X˙=F(X)\dot X = F(X) with FF continuously differentiable, carry a bounded region D(0)D(0) of initial states along the solutions, let D(t)D(t) be the region they fill at time tt, and let V(t)V(t) be its volume. While these solutions exist,

V˙(t)=∫D(t)∇⋅F(X) dX,\dot V(t) = \int_{D(t)} \nabla \cdot F(X)\,dX,

where dXdX is the volume element. So a region loses volume while the divergence ∇⋅F\nabla \cdot F is negative throughout it, and the flow contracts volume when the divergence is negative everywhere. For the Lorenz field the divergence is the constant −(σ+1+β)-(\sigma + 1 + \beta), and therefore

V(t)=V(0) e−(σ+1+β)t.V(t) = V(0)\,e^{-(\sigma + 1 + \beta)t}.

In plain words The system couples three rates through just two product terms, and it has a built-in squeeze. Any blob of starting states loses volume at the same exponential rate wherever it is, even while it is being stretched in some direction.

Example Take the classical values. At an equilibrium the first equation gives y=xy = x, and the second then reads x(27−z)=0x(27 - z) = 0. Both cases must be kept, so do not divide by xx. If x=0x = 0, then y=0y = 0 and the third equation gives z=0z = 0, the origin. If x≠0x \ne 0, then z=27z = 27 and the third equation becomes x2=83⋅27=72x^2 = \tfrac{8}{3} \cdot 27 = 72, so x=y=±62x = y = \pm 6\sqrt{2}. Substituting (62,62,27)(6\sqrt{2}, 6\sqrt{2}, 27) gives 10⋅0=010 \cdot 0 = 0, then 62⋅1−62=06\sqrt{2} \cdot 1 - 6\sqrt{2} = 0, then 72−72=072 - 72 = 0, and changing the signs of xx and yy together leaves all three checks intact. At the origin the Jacobian matrix is

J(0)=[−1010028−1000−83].J(0) = \begin{bmatrix} -10 & 10 & 0 \\ 28 & -1 & 0 \\ 0 & 0 & -\frac{8}{3} \end{bmatrix}.

Its upper left block has characteristic polynomial λ2+11λ−270\lambda^2 + 11\lambda - 270, so the eigenvalues are

λ1,2=−11±12012,λ3=−83,\begin{gathered} \lambda_{1,2} = \frac{-11 \pm \sqrt{1201}}{2}, \\[0.4em] \lambda_3 = -\frac{8}{3}, \end{gathered}

about 11.8311.83, −22.83-22.83 and −2.67-2.67. One direction stretches, so the origin is unstable. The divergence is −10−1−83=−413-10 - 1 - \tfrac{8}{3} = -\tfrac{41}{3}, which is also the sum of these three eigenvalues, and volumes shrink like e−41t/3e^{-41t/3}, by a factor of about 1.2×10−61.2 \times 10^{-6} in one time unit. The stretching in one direction is outweighed by stronger squeezing in the others.

Attractor

Definition Let the solutions of X˙=F(X)\dot X = F(X), with FF continuously differentiable, exist for all t≥0t \ge 0, and write φt\varphi_t for the flow. A closed bounded set AA of states is an attractor if three conditions hold. First, the set is invariant, meaning φt(A)=A\varphi_t(A) = A for every t≥0t \ge 0. Second, it attracts an open set UU containing it, so every solution that starts in UU has distance from AA tending to 00 as t→∞t \to \infty. Third, it is minimal, so no smaller closed subset of AA satisfies the first two conditions. The set of all initial states whose solutions approach AA is its basin of attraction. Definitions in the literature differ in the third condition, for example by asking for a solution that comes arbitrarily close to every point of AA, so the one in use should be named. An attractor on which the motion is chaotic is called a chaotic attractor, and often a strange attractor. If ∇⋅F≤−c\nabla \cdot F \le -c everywhere for a constant c>0c > 0, every closed bounded invariant set has zero volume, because its volume VV satisfies V≤e−ctVV \le e^{-ct} V for every t>0t > 0.

In plain words An attractor is where the long-term motion lives. It is a bounded set that solutions never leave once inside and that pulls in every start from a whole neighborhood. It can be a single point, a closed orbit or something far more intricate.

Example The damped oscillator q¨+q˙+q=0\ddot q + \dot q + q = 0, written as q˙=v\dot q = v, v˙=−q−v\dot v = -q - v, has divergence −1-1, so areas shrink like e−te^{-t}. Its eigenvalues (−1±i3)/2(-1 \pm i\sqrt{3})/2 have negative real part, every solution tends to the origin, and the origin alone is an attractor whose basin is the whole plane. Here contraction ends in rest, not in chaos. For the Lorenz system, the function W=ρx2+σy2+σ(z−2ρ)2W = \rho x^2 + \sigma y^2 + \sigma(z - 2\rho)^2 changes along solutions at the rate

W˙=−2σ (Q−βρ2),Q=ρx2+y2+β(z−ρ)2,\begin{gathered} \dot W = -2\sigma\,(Q - \beta\rho^2), \\[0.4em] Q = \rho x^2 + y^2 + \beta(z - \rho)^2, \end{gathered}

which is negative outside the bounded ellipsoid Q≤βρ2Q \le \beta\rho^2. From this one shows that every solution exists for all t≥0t \ge 0 and eventually enters a fixed bounded region and stays there. Since the classical divergence is the constant −41/3-41/3, every attractor of the classical system has zero volume. The butterfly-shaped set that computed solutions trace out has no volume at all, however solid it looks in a picture.

Largest Lyapunov exponent

Definition Let X(t)X(t) be a bounded solution of X˙=F(X)\dot X = F(X) for t≥0t \ge 0, with FF continuously differentiable and Jacobian matrix J(X)J(X). To first order, a perturbation ww of the state evolves by the variational equation w˙=J(X(t)) w\dot w = J(X(t))\,w. Let Φ(t)\Phi(t) be its fundamental matrix with Φ(0)=I\Phi(0) = I. The largest Lyapunov exponent of the solution is

λmax⁡=lim sup⁡t→∞1tln⁡∥Φ(t)∥.\lambda_{\max} = \limsup_{t \to \infty} \frac{1}{t} \ln \lVert \Phi(t) \rVert.

Every perturbation w(0)≠0w(0) \ne 0 has the exponential growth rate lim sup⁡t→∞1tln⁡∣w(t)∣\limsup_{t \to \infty} \frac{1}{t} \ln \lvert w(t) \rvert, which is at most λmax⁡\lambda_{\max}, and every perturbation outside a proper subspace has exactly the rate λmax⁡\lambda_{\max}. The number belongs to the solution being followed, and it describes infinitesimal perturbations over infinite time. A computation follows only a finite time TT with finite accuracy, so it returns a finite-time estimate

λT=1Tln⁡∣w(T)∣∣w(0)∣,\lambda_T = \frac{1}{T} \ln \frac{\lvert w(T) \rvert}{\lvert w(0) \rvert},

usually obtained by rescaling the perturbation at regular intervals, adding the logarithms of the growth factors, and discarding an initial transient.

In plain words A positive exponent means that a typical tiny error grows roughly like eλmax⁡te^{\lambda_{\max} t} along that solution. It measures how fast detailed predictions fail, but a value computed from a finite run is evidence, not a theorem.

Example At the origin of the classical Lorenz system the solution is constant, Φ(t)=eJ(0)t\Phi(t) = e^{J(0)t}, and the exponent is the largest real part of an eigenvalue of J(0)J(0), namely (−11+1201)/2≈11.83(-11 + \sqrt{1201})/2 \approx 11.83. Every perturbation with a nonzero component along the unstable eigenvector grows at this rate. The exponent is positive, yet it only records that the origin is an unstable equilibrium, which is not chaos. Along solutions that wind around the Lorenz attractor, long computations give estimates near 0.9060.906. Runs of 20002000 time units with classical Runge-Kutta steps of 0.010.01 and 0.0050.005, and from a second start, gave 0.9060.906, 0.9100.910 and 0.9120.912, so even the second decimal is uncertain after that much work. Two runs from the same start with those two step sizes are in completely different places on the attractor by t=60t = 60, because every step error acts like a tiny change of the state. Late coordinates of such runs are not reliable, while averaged quantities such as the exponent still agree to within about 0.010.01.

Chaos

Definition Chaos has several precise definitions in use, so a claim of chaos should name the one it relies on. A common working definition calls a deterministic system chaotic when three things hold. It has bounded solutions that never settle onto an equilibrium, a periodic orbit or a quasiperiodic motion. These solutions start from a set of initial states that is not negligible, such as an open set. Finally, nearby solutions separate in the sense of sensitive dependence on initial conditions, which in practice is detected by a positive largest Lyapunov exponent. A stricter definition concerns the motion on a closed bounded invariant set Λ\Lambda and asks for sensitive dependence on Λ\Lambda, a solution in Λ\Lambda that passes arbitrarily close to every point of Λ\Lambda, and periodic orbits passing arbitrarily close to every point of Λ\Lambda. Smooth planar autonomous systems leave no room for chaos. By the Poincaré-Bendixson theorem, if X˙=F(X)\dot X = F(X) in R2\mathbb{R}^2 has FF continuously differentiable and only finitely many equilibria, the omega-limit set of a solution that stays bounded for t≥0t \ge 0 is an equilibrium, a periodic orbit, or equilibria joined by solutions that tend to equilibria as t→±∞t \to \pm\infty, and none of these carries chaotic motion. Chaos in a smooth autonomous system therefore needs at least three state variables. Three variables make chaos possible but never guarantee it.

In plain words Chaotic motion is fully determined by its equations, stays in a bounded region, never settles into a repeating pattern, and amplifies small differences until they matter. Unstable equilibria, shrinking volumes and a tangled picture are all consistent with chaos, but none of them proves it.

Example For the classical Lorenz system three facts are exact. All three equilibria are unstable, since the origin has the eigenvalue 11.8311.83 and each of C±C_{\pm} has the complex pair 0.094±10.19i0.094 \pm 10.19i. Every solution enters a bounded region. Volumes shrink like e−41t/3e^{-41t/3}. Numerical evidence adds nearby solutions that separate and a largest Lyapunov exponent estimated near 0.9060.906. Neither level is a proof of chaos. With ρ=350\rho = 350 and the other parameters unchanged, all three equilibria are still unstable and the divergence is still −41/3-41/3, yet a numerical solution from (1,1,1)(1, 1, 1) settles onto a periodic orbit. For the classical values, the existence of a chaotic attractor was proved in 2002 by a computer-assisted argument with rigorous error bounds, which is a different kind of statement from both the exact checks and the estimates.

Forced Duffing equation

Definition With tt as time and a dot for d/dtd/dt, the forced Duffing equation for a displacement q(t)q(t) is

q¨+δq˙+αq+βq3=γcos⁡(ωt),\begin{aligned} &\ddot q + \delta \dot q + \alpha q + \beta q^3 \\[0.4em] &\qquad = \gamma \cos(\omega t), \end{aligned}

with damping δ≥0\delta \ge 0, linear stiffness α\alpha, cubic stiffness β\beta, forcing amplitude γ\gamma and forcing angular frequency ω>0\omega > 0. With the velocity v=q˙v = \dot q and γ≠0\gamma \ne 0, the pair (q,v)(q, v) obeys a planar system whose right side depends on tt, so the planar theorems for autonomous systems do not apply to it. Taking the forcing phase θ\theta as a third state, read modulo 2π2\pi, gives the autonomous system

q˙=v,v˙=−δv−αq−βq3+γcos⁡θ,θ˙=ω.\begin{aligned} \dot q &= v, \\[0.4em] \dot v &= -\delta v - \alpha q - \beta q^3 \\ &\qquad + \gamma \cos\theta, \\[0.4em] \dot \theta &= \omega. \end{aligned}

A solution q(t)q(t) of the forced equation corresponds to the solution (q(t),q˙(t),ωt)(q(t), \dot q(t), \omega t) of this system, the one with θ(0)=0\theta(0) = 0. The divergence of the new field is −δ-\delta, so for δ>0\delta > 0 the three-dimensional flow shrinks volume like e−δte^{-\delta t}.

In plain words Position and velocity are not enough to predict a forced oscillator, because the future also depends on where the forcing is in its cycle. Recording that phase as a third coordinate makes the system autonomous and three-dimensional, which is exactly the case the planar restriction on chaos does not cover.

Example Take δ=α=γ=ω=1\delta = \alpha = \gamma = \omega = 1 and β=0\beta = 0, the linear equation q¨+q˙+q=cos⁡t\ddot q + \dot q + q = \cos t. The function q=sin⁡tq = \sin t solves it, since −sin⁡t+cos⁡t+sin⁡t=cos⁡t-\sin t + \cos t + \sin t = \cos t. In the three-dimensional form this solution is (q,v,θ)=(sin⁡t,cos⁡t,t)(q, v, \theta) = (\sin t, \cos t, t). Indeed v˙=−sin⁡t\dot v = -\sin t equals −v−q+cos⁡θ=−cos⁡t−sin⁡t+cos⁡t-v - q + \cos\theta = -\cos t - \sin t + \cos t, and θ˙=1\dot\theta = 1. Every other solution differs from this one by a term

e−t/2(c1cos⁡3 t2+c2sin⁡3 t2),e^{-t/2}\left(c_1 \cos\tfrac{\sqrt{3}\,t}{2} + c_2 \sin\tfrac{\sqrt{3}\,t}{2}\right),

so all solutions approach the same periodic motion and nothing here is chaotic. Forcing alone does not create chaos. With the double-well values α=−1\alpha = -1, β=1\beta = 1, δ=0.3\delta = 0.3 and ω=1.2\omega = 1.2, a computation with γ=0.2\gamma = 0.2 from (q,v)=(1,0)(q, v) = (1, 0) settles onto a response that repeats with the forcing period. With γ=0.5\gamma = 0.5 the states sampled once per forcing period keep wandering, and long runs estimate a largest Lyapunov exponent near 0.10.1. That is evidence of chaos at γ=0.5\gamma = 0.5, not a proof.

Check yourself

  1. Find every equilibrium of the Lorenz system with σ=10\sigma = 10, β=8/3\beta = 8/3 and ρ=10\rho = 10, and check one of them. What changes when ρ=1/2\rho = 1/2?
Show the answer

At an equilibrium the first equation gives y=xy = x, and the second becomes x(10−z)−x=x(9−z)=0x(10 - z) - x = x(9 - z) = 0. If x=0x = 0, then y=0y = 0 and the third equation −83z=0-\tfrac{8}{3}z = 0 gives z=0z = 0. If x≠0x \ne 0, then z=9z = 9 and the third equation gives x2=83⋅9=24x^2 = \tfrac{8}{3} \cdot 9 = 24, so x=y=±26x = y = \pm 2\sqrt{6}. The equilibria are the origin and (±26,±26,9)(\pm 2\sqrt{6}, \pm 2\sqrt{6}, 9) with matching signs. For (26,26,9)(2\sqrt{6}, 2\sqrt{6}, 9) the three components are 10⋅0=010 \cdot 0 = 0, then 26(10−9)−26=02\sqrt{6}(10 - 9) - 2\sqrt{6} = 0, then 24−24=024 - 24 = 0.

With ρ=1/2\rho = 1/2 the second equation becomes x(−12−z)=0x(-\tfrac{1}{2} - z) = 0. The case x≠0x \ne 0 would need z=−12z = -\tfrac{1}{2} and then x2=83z=−43x^2 = \tfrac{8}{3}z = -\tfrac{4}{3}, which is impossible. Only the origin remains, as the condition ρ>1\rho > 1 for the two outer equilibria predicts.

  1. A classmate argues that the divergence of the classical Lorenz field is −41/3-41/3, so nearby solutions must move toward each other and sensitive dependence is impossible. Explain the mistake using the system x˙=x\dot x = x, y˙=−2y\dot y = -2y.
Show the answer

This system has divergence 1−2=−11 - 2 = -1, so areas shrink like e−te^{-t}. Its solutions are x=x0etx = x_0 e^{t} and y=y0e−2ty = y_0 e^{-2t}. A small square of side dd becomes a rectangle with sides detd e^{t} and de−2td e^{-2t}, whose area d2e−td^2 e^{-t} does shrink. Yet the two starts (x0,y0)(x_0, y_0) and (x0+d,y0)(x_0 + d, y_0) are detd e^{t} apart at time tt, however small dd is.

A negative divergence controls the product of the stretching factors, not each factor separately. It does not imply boundedness either, since every start with x0≠0x_0 \ne 0 runs off to infinity. The Lorenz origin shows the same split, with the rate 11.8311.83 in one direction and the total −41/3-41/3.

  1. With tt as time, explain why the unforced equation q¨+0.3q˙−q+q3=0\ddot q + 0.3\dot q - q + q^3 = 0 cannot behave chaotically. Then write the forced equation q¨+0.3q˙−q+q3=0.5cos⁡(1.2t)\ddot q + 0.3\dot q - q + q^3 = 0.5\cos(1.2t) as an autonomous system, and say what the change of dimension allows and what it does not prove.
Show the answer

With v=q˙v = \dot q the unforced equation is the planar autonomous system q˙=v\dot q = v, v˙=q−q3−0.3v\dot v = q - q^3 - 0.3v. Its right side is a polynomial, and its equilibria are (0,0)(0, 0) and (±1,0)(\pm 1, 0). The energy

E=v22−q22+q44E = \frac{v^2}{2} - \frac{q^2}{2} + \frac{q^4}{4}

satisfies E˙=v(q−q3−0.3v)+(q3−q)v=−0.3v2≤0\dot E = v(q - q^3 - 0.3v) + (q^3 - q)v = -0.3v^2 \le 0, and it grows without bound as ∣(q,v)∣→∞\lvert (q, v) \rvert \to \infty, so every solution stays in a bounded set for t≥0t \ge 0. By the Poincaré-Bendixson theorem each omega-limit set is an equilibrium, a periodic orbit, or equilibria joined by connecting solutions, and none of these is chaotic. In fact the energy strictly decreases along every nonconstant solution, which rules out periodic orbits, and every solution tends to one of the three equilibria.

The forced equation becomes

q˙=v,v˙=q−q3−0.3v+0.5cos⁡θ,θ˙=1.2,\begin{aligned} \dot q &= v, \\[0.4em] \dot v &= q - q^3 - 0.3v \\ &\qquad + 0.5\cos\theta, \\[0.4em] \dot\theta &= 1.2, \end{aligned}

with θ\theta read modulo 2π2\pi. This system is three-dimensional, so the planar theorem no longer applies and chaos becomes possible. The extra dimension proves nothing by itself. With 0.20.2 in place of 0.50.5, a numerical solution from (q,v)=(1,0)(q, v) = (1, 0) settles onto a periodic response, and at 0.50.5 a positive exponent estimate near 0.10.1 is evidence of chaos rather than a proof.

How this lesson connects