Reference sheet

Differential Equations reference sheet

The main definitions, formulas, theorems and conditions of the differential equations course on one page, unit by unit, with links to the lessons that explain each topic in full.

For Differential Equations

This sheet follows the thirteen units of the course in order. Each unit starts with the big idea in a few sentences, then gives the definitions, formulas, theorems and methods you will look up most often, with the conditions they need. When you want the full explanation, a derivation or more examples, follow the lesson links under each topic.

Unless a topic says otherwise, the independent variable is tt and a prime is a derivative with respect to it, so y′=dy/dty' = dy/dt. Topics about space or power series use xx and say so. A scalar unknown is yy, a state vector is a plain capital letter such as X=(x1,…,xn)TX = (x_1, \ldots, x_n)^T, and matrices are plain capitals such as AA, II and Φ\Phi. Partial derivatives are written ut=∂u/∂tu_t = \partial u/\partial t and uxx=∂2u/∂x2u_{xx} = \partial^2 u/\partial x^2, and the only dot is in V˙\dot V, the rate of change of a function VV along solutions. The order of an equation is its highest derivative, which is a different thing from the degree of a polynomial. Logarithms are natural logarithms, and the conventions for the Laplace transform, Fourier series and the Fourier transform are stated where they are used. A formula holds wherever both sides are defined unless a condition says otherwise.

Language, Solutions and Modeling

A differential equation relates an unknown function to its derivatives. A solution is a function that satisfies it at every point of an interval, and initial or boundary data pick one solution out of a family. Before using any method, identify the kind of equation, the data and the interval, and check whether a solution is guaranteed to exist and to be unique.

Classifying an equation

An ordinary differential equation (ODE) has one independent variable, and a partial differential equation (PDE) has several and involves partial derivatives. The order is the highest derivative that appears, whatever power it is raised to. An equation of order nn is linear when it can be written as

an(t) y(n)+⋯+a1(t) y′+a0(t) y=g(t),\begin{aligned} a_n(t)\,y^{(n)} &+ \cdots + a_1(t)\,y' \\[0.4em] &+ a_0(t)\,y = g(t), \end{aligned}

with coefficients aka_k and forcing gg that depend on tt only. It is homogeneous when g=0g = 0 and nonhomogeneous otherwise. Powers such as y2y^2 or (y′)2(y')^2, products such as y y′y\,y' and functions such as sin⁡y\sin y make an equation nonlinear. A first-order equation y′=F(y/x)y' = F(y/x) is also called homogeneous, in a second sense that is about scaling, not about forcing.

EquationTypeOrderStructure
y′′+3y′+2y=0y'' + 3y' + 2y = 0ODE22linear, homogeneous
y′+2y=sin⁡ty' + 2y = \sin tODE11linear, nonhomogeneous
y′′+(cos⁡t) y=0y'' + (\cos t)\,y = 0ODE22linear with a variable coefficient
(y′)2+y3=t(y')^2 + y^3 = tODE11nonlinear
ut=κ uxxu_t = \kappa\,u_{xx}PDE22linear, homogeneous

Lessons Differential equations and their language

Solutions and side conditions

A solution of an equation of order nn on an open interval II is a function with nn derivatives on II that satisfies the equation at every point of II. It is explicit when it is given as a formula for yy and implicit when a relation G(t,y)=0G(t, y) = 0 defines yy as a differentiable function on II. A general solution is a family with arbitrary constants, usually nn of them, and a particular solution is one member of the family.

  • Initial value problem. The values y(t0),y′(t0),…,y(n−1)(t0)y(t_0), y'(t_0), \ldots, y^{(n-1)}(t_0) are given at one point.
  • Boundary value problem. Conditions are given at two or more points. A linear one has exactly one solution, none or infinitely many.

In the variable xx, the equation y′′+y=0y'' + y = 0 has the general solution C1cos⁡x+C2sin⁡xC_1\cos x + C_2\sin x. The initial values y(0)=2y(0) = 2 and y′(0)=−1y'(0) = -1 select 2cos⁡x−sin⁡x2\cos x - \sin x. The boundary conditions y(0)=y(π)=0y(0) = y(\pi) = 0 allow every Csin⁡xC\sin x, while y(0)=0y(0) = 0 and y(π)=1y(\pi) = 1 allow none. The relation x2+y2=25x^2 + y^2 = 25 gives the explicit solution y=25−x2y = \sqrt{25 - x^2} of y′=−x/yy' = -x/y only on −5<x<5-5 < x < 5, since y=0y = 0 at the endpoints. To verify a solution, substitute it into the equation and the data and state the interval.

Lessons Solutions and conditions

Calculus and complex tools

The product rule (uv)′=u′v+uv′(uv)' = u'v + uv' and the chain rule ddtf(y(t))=f′(y(t)) y′(t)\frac{d}{dt}f(y(t)) = f'(y(t))\,y'(t) are used to check every solution. If ff is continuous on an interval containing t0t_0, the solutions of y′=f(t)y' = f(t) there are

y(t)=y(t0)+∫t0tf(s) ds,y(t) = y(t_0) + \int_{t_0}^{t} f(s)\,ds,

and an indefinite integral carries one arbitrary constant on each interval. In particular ∫dtt=ln⁡∣t∣+C\int \frac{dt}{t} = \ln\lvert t \rvert + C on each interval that does not contain 00.

A complex number z=a+ibz = a + ib with i2=−1i^2 = -1 has real part aa, imaginary part bb, conjugate z‾=a−ib\overline{z} = a - ib and modulus ∣z∣=a2+b2\lvert z \rvert = \sqrt{a^2 + b^2}. The nonreal roots of a polynomial with real coefficients come in conjugate pairs. For real θ\theta, aa and bb, Euler’s formula and the complex exponential are

eiθ=cos⁡θ+isin⁡θ,e(a+ib)t=eateibt=eat(cos⁡bt+isin⁡bt).\begin{aligned} &e^{i\theta} = \cos\theta + i\sin\theta, \\[0.4em] &e^{(a + ib)t} = e^{at}e^{ibt} \\[0.4em] &\quad = e^{at}(\cos bt + i\sin bt). \end{aligned}

For every complex constant rr, ddtert=rert\frac{d}{dt}e^{rt} = re^{rt}. If a complex function solves a linear equation with real coefficients, its real and imaginary parts are real solutions. For example (−1+2i)2+2(−1+2i)+5=0(-1 + 2i)^2 + 2(-1 + 2i) + 5 = 0, so the real part e−tcos⁡2te^{-t}\cos 2t of e(−1+2i)te^{(-1 + 2i)t} solves y′′+2y′+5y=0y'' + 2y' + 5y = 0.

Lessons Calculus and complex numbers

Direction fields

For y′=f(x,y)y' = f(x, y) in the variable xx, the direction field draws at each point (x,y)(x, y) a short segment of slope f(x,y)f(x, y). A solution curve is tangent to the field at every one of its points. An isocline f(x,y)=mf(x, y) = m is a curve along which all segments have the same slope mm, and in general it is not a solution curve. For y′=x−yy' = x - y the isoclines are the lines y=x−my = x - m and the solutions are y=x−1+Ce−xy = x - 1 + Ce^{-x}. The isocline m=1m = 1, the line y=x−1y = x - 1, happens to be the solution with C=0C = 0, and every other solution approaches it as x→∞x \to \infty.

Lessons Direction fields

Existence and uniqueness

Consider y′=f(t,y)y' = f(t, y) with y(t0)=y0y(t_0) = y_0.

  • Existence. If ff is continuous on an open rectangle containing (t0,y0)(t_0, y_0), at least one solution exists on some open interval containing t0t_0. The interval may be shorter than the rectangle.
  • Uniqueness. If moreover ff is locally Lipschitz in yy, meaning that near each point some constant LL gives ∣f(t,y1)−f(t,y2)∣≤L ∣y1−y2∣\lvert f(t, y_1) - f(t, y_2) \rvert \le L\,\lvert y_1 - y_2 \rvert, the solution is unique. A continuous ∂f/∂y\partial f/\partial y is enough for this but not necessary, since f=∣y∣f = \lvert y \rvert is Lipschitz.
  • Maximal interval. Under both hypotheses on an open set DD, the solution extends to a largest open interval of existence (α,ω)(\alpha, \omega). If ω<∞\omega < \infty, the point (t,y(t))(t, y(t)) leaves every closed and bounded subset of DD as t→ω−t \to \omega^-, for example by blowing up, and the same holds at α\alpha.

The theorems are local. The function f(t,y)=y2f(t, y) = y^2 is smooth everywhere, yet y′=y2y' = y^2 with y(0)=1y(0) = 1 has the solution y=11−ty = \frac{1}{1 - t} only on (−∞,1)(-\infty, 1), which is finite-time blowup. The equation y′=2∣y∣y' = 2\sqrt{\lvert y \rvert} with y(0)=0y(0) = 0 has a continuous right side that is not Lipschitz at y=0y = 0, and it has infinitely many solutions, the zero solution and, for each a≥0a \ge 0,

ya(t)={0,t≤a,(t−a)2,t≥a.y_a(t) = \begin{cases} 0, & t \le a, \\ (t - a)^2, & t \ge a. \end{cases}

Lessons Existence and uniqueness

Models, units and scaling

A balance law says that the rate of change of an amount equals what flows in minus what flows out, plus what is produced minus what is used up. Every term has the same units, those of the amount per unit time, and every exponent or argument of a function such as e−kte^{-kt} is dimensionless. The state is the quantity the equation evolves, and the parameters are fixed numbers that specify the model. To nondimensionalize, divide the state and the time by characteristic scales, u=y/Yu = y/Y and τ=t/T\tau = t/T, chosen so that fewer parameters remain.

For a tank holding V=10V = 10 liters, with inflow and outflow of 22 liters per minute, inflow concentration 33 grams per liter and M(0)=10M(0) = 10 grams of salt, the amount MM in grams after tt minutes satisfies

dMdt=6−M5,M(t)=30−20e−t/5.\begin{gathered} \frac{dM}{dt} = 6 - \frac{M}{5}, \\[0.4em] M(t) = 30 - 20e^{-t/5}. \end{gathered}

The scales u=M/30u = M/30 and τ=t/5\tau = t/5 give dudτ=1−u\frac{du}{d\tau} = 1 - u with u(0)=13u(0) = \frac{1}{3}, an equation with no parameters left.

Lessons Modeling and units

First-Order Solution Methods

Many first-order equations can be solved exactly once their form is recognized. Each method below changes the equation into one that can be integrated, and each can lose solutions or hide the interval where a solution is valid, so those are checked after solving.

Separable equations

An equation y′=a(t) b(y)y' = a(t)\,b(y) is separable. Take aa continuous and bb continuously differentiable, so that solutions are unique.

  1. Find the constant solutions y=cy = c with b(c)=0b(c) = 0 first, because dividing by b(y)b(y) loses them.
  2. Where b(y)≠0b(y) \ne 0, integrate both sides of dyb(y)=a(t) dt\frac{dy}{b(y)} = a(t)\,dt.
  3. Solve for yy if possible, apply the initial value and state the interval where the solution is defined.

By uniqueness a nonconstant solution never reaches a zero of bb, so the constant solutions and the separated ones do not mix. For y′=2tyy' = 2ty the constant solution is y=0y = 0, and elsewhere ln⁡∣y∣=t2+C0\ln\lvert y \rvert = t^2 + C_0. Together they give y=Cet2y = Ce^{t^2} for every real CC, and y(0)=3y(0) = 3 selects y=3et2y = 3e^{t^2}. An implicit answer is often the natural one. In the variable xx, the equation y y′=−xy\,y' = -x gives x2+y2=Cx^2 + y^2 = C, and with y(0)=2y(0) = 2 the solution is y=4−x2y = \sqrt{4 - x^2} on −2<x<2-2 < x < 2.

Lessons Separable equations

Linear equations and integrating factors

Divide a(t) y′+b(t) y=g(t)a(t)\,y' + b(t)\,y = g(t) by a(t)a(t) on an interval where a(t)≠0a(t) \ne 0 to get the standard form y′+p(t) y=q(t)y' + p(t)\,y = q(t). The integrating factor μ=e∫p dt\mu = e^{\int p\,dt} satisfies μ′=pμ\mu' = p\mu, which turns the left side into a product derivative, (μy)′=μq(\mu y)' = \mu q.

If pp and qq are continuous on an open interval II containing t0t_0, the problem with y(t0)=y0y(t_0) = y_0 has exactly one solution, it exists on all of II, and

μ(t) y(t)=y0+∫t0tμ(s) q(s) ds,μ(t)=exp⁡(∫t0tp(s) ds).\begin{aligned} &\mu(t)\,y(t) = y_0 \\[0.4em] &\quad + \int_{t_0}^{t} \mu(s)\,q(s)\,ds, \\[0.4em] &\mu(t) = \exp\left(\int_{t_0}^{t} p(s)\,ds\right). \end{aligned}

For y′+2y=4ty' + 2y = 4t with y(0)=3y(0) = 3, the factor e2te^{2t} gives (e2ty)′=4te2t(e^{2t}y)' = 4te^{2t} and y=2t−1+4e−2ty = 2t - 1 + 4e^{-2t}. Points where the leading coefficient vanishes split the line. In the variable xx, the equation x y′+y=1x\,y' + y = 1 has the solutions y=1+C/xy = 1 + C/x, each defined on x>0x > 0 or on x<0x < 0 but never across x=0x = 0.

Lessons Linear first-order equations

Exact equations and integrating factors

The equation M(x,y) dx+N(x,y) dy=0M(x, y)\,dx + N(x, y)\,dy = 0, with MM and NN continuously differentiable on an open set DD, is exact on DD when it has a potential function FF there, with Fx=MF_x = M and Fy=NF_y = N. Its solutions lie on the level curves F(x,y)=CF(x, y) = C, and where N≠0N \ne 0 they solve y′=−M/Ny' = -M/N.

  • Exactness on DD implies My=NxM_y = N_x on DD.
  • If DD is simply connected, such as a disk or a rectangle, then My=NxM_y = N_x on DD implies exactness. So the test always works locally.
  • On a region with a hole it can fail. The form −y dx+x dyx2+y2\frac{-y\,dx + x\,dy}{x^2 + y^2} satisfies My=NxM_y = N_x on the plane without the origin, but its integral around the unit circle is 2π2\pi, so it has no potential there.

To find a potential, integrate MM in xx to get F=∫M dx+h(y)F = \int M\,dx + h(y), then choose hh so that Fy=NF_y = N. For (2xy+3x2) dx+(x2+4y) dy=0(2xy + 3x^2)\,dx + (x^2 + 4y)\,dy = 0 the test gives My=Nx=2xM_y = N_x = 2x, and the potential is F=x2y+x3+2y2F = x^2y + x^3 + 2y^2.

A nonexact equation can become exact after multiplication by an integrating factor μ\mu. Two tests look for one that depends on a single variable.

FactorWorks when this quotientFormula
μ(x)\mu(x)r=My−NxNr = \dfrac{M_y - N_x}{N} depends on xx onlyμ=e∫r(x) dx\mu = e^{\int r(x)\,dx}
μ(y)\mu(y)r=Nx−MyMr = \dfrac{N_x - M_y}{M} depends on yy onlyμ=e∫r(y) dy\mu = e^{\int r(y)\,dy}

The new equation has the same solutions only where μ\mu is defined and nonzero, so state that region. For y dx+2x dy=0y\,dx + 2x\,dy = 0 on x>0x > 0, the first quotient is −12x-\frac{1}{2x}, so μ=x−1/2\mu = x^{-1/2}, the potential is F=2yxF = 2y\sqrt{x} and the solutions are yx=Cy\sqrt{x} = C.

Lessons Exact equations, Integrating factors

Substitutions

EquationSubstitutionResult
Homogeneous, y′=F ⁣(yx)y' = F\!\left(\dfrac{y}{x}\right)y=xvy = xv, so y′=v+xv′y' = v + xv'separable, xv′=F(v)−vxv' = F(v) - v
Bernoulli, y′+p(t) y=q(t) yny' + p(t)\,y = q(t)\,y^n with n≠0,1n \ne 0, 1z=y1−nz = y^{1-n}linear, z′+(1−n) p z=(1−n) qz' + (1 - n)\,p\,z = (1 - n)\,q
Riccati, y′=a(t) y2+b(t) y+c(t)y' = a(t)\,y^2 + b(t)\,y + c(t) with a known solution ypy_py=yp+1uy = y_p + \dfrac{1}{u}linear, u′+(2a yp+b) u=−au' + (2a\,y_p + b)\,u = -a

Each substitution misses some solutions, which have to be added back.

  • Homogeneous. Every root v0v_0 of F(v)=vF(v) = v gives a line solution y=v0xy = v_0x that is lost when dividing by F(v)−vF(v) - v, and x=0x = 0 is excluded. With x>0x > 0, the problem y′=1+y/xy' = 1 + y/x, y(1)=0y(1) = 0 gives xv′=1xv' = 1 and y=xln⁡xy = x\ln x.
  • Bernoulli. For n>0n > 0 the zero function is a solution that the new variable misses, and for fractional nn the solution must stay where yny^n is defined. For y′+y=y2y' + y = y^2 with y(0)=12y(0) = \frac{1}{2}, the variable z=1/yz = 1/y satisfies z′−z=−1z' - z = -1, so z=1+etz = 1 + e^t and y=11+ety = \frac{1}{1 + e^t}.
  • Riccati. The known solution ypy_p itself corresponds to no finite uu, and yy has a pole wherever u=0u = 0. For y′=y2−1y' = y^2 - 1 with yp=1y_p = 1 and y(0)=0y(0) = 0, the correction satisfies u′+2u=−1u' + 2u = -1 with u(0)=−1u(0) = -1, so u=−12(1+e−2t)u = -\frac{1}{2}(1 + e^{-2t}) and y=1+1u=−tanh⁡ty = 1 + \frac{1}{u} = -\tanh t.

Lessons Homogeneous equations, Bernoulli equations, Riccati equations

First-Order Models and Qualitative Behavior

An autonomous equation can be understood from the sign of its right side, without a formula for the solution. The models in this unit, growth and decay, cooling, logistic growth with harvesting, mixing and falling with drag, are first-order equations whose parameters carry units and whose solutions make sense only on a physical domain.

Equilibria and the phase line

For an autonomous equation y′=f(y)y' = f(y) with ff continuously differentiable, an equilibrium is a number y∗y_* with f(y∗)=0f(y_*) = 0, and y=y∗y = y_* is then a constant solution. Between two equilibria ff keeps one sign, so a solution there increases where f>0f > 0, decreases where f<0f < 0 and never reaches an equilibrium in finite time. The phase line marks the equilibria on the yy axis and draws arrows in these directions.

  • Stable. For every ε>0\varepsilon > 0 there is δ>0\delta > 0 such that ∣y(0)−y∗∣<δ\lvert y(0) - y_* \rvert < \delta gives ∣y(t)−y∗∣<ε\lvert y(t) - y_* \rvert < \varepsilon for all t≥0t \ge 0.
  • Asymptotically stable. Stable, and every solution that starts close enough tends to y∗y_*.
  • Semistable. Solutions approach y∗y_* from one side and move away on the other, so the equilibrium is unstable.
Condition at y∗y_*Conclusion
f′(y∗)<0f'(y_*) < 0asymptotically stable
f′(y∗)>0f'(y_*) > 0unstable
f′(y∗)=0f'(y_*) = 0no conclusion, as y′=−y3y' = -y^3, y′=y3y' = y^3 and y′=y2y' = y^2 show at 00

For y′=y(2−y)y' = y(2 - y) the equilibrium 00 is unstable and 22 is asymptotically stable, and y(0)=1y(0) = 1 gives y=21+e−2ty = \frac{2}{1 + e^{-2t}}, which rises toward 22. The equation y′=(y−1)2y' = (y - 1)^2 is semistable at 11. From y(0)=0y(0) = 0 the solution 1−11+t1 - \frac{1}{1 + t} approaches 11, while from y(0)=2y(0) = 2 the solution 1+11−t1 + \frac{1}{1 - t} blows up at t=1t = 1.

Lessons Phase-line stability

Growth, decay and cooling

ModelEquationSolutionKey quantities
GrowthQ′=kQQ' = kQ, k>0k > 0Q0ektQ_0e^{kt}doubling time ln⁡2k\dfrac{\ln 2}{k}
DecayQ′=−λQQ' = -\lambda Q, λ>0\lambda > 0Q0e−λtQ_0e^{-\lambda t}half-life ln⁡2λ\dfrac{\ln 2}{\lambda}, time constant 1λ\dfrac{1}{\lambda}
Newton coolingT′=−k(T−A)T' = -k(T - A), k>0k > 0, constant AAA+(T0−A) e−ktA + (T_0 - A)\,e^{-kt}the difference T−AT - A decays

After one time constant a decaying amount is Q0/eQ_0/e, about 37 percent of its starting value. A half-life of 44 hours gives λ=ln⁡24\lambda = \frac{\ln 2}{4} per hour, so 8080 grams decays to 4040 grams after 44 hours and to 2020 grams after 88 hours. A cup at 9090 degrees in a room at A=20A = 20 degrees with k=15k = \frac{1}{5} per minute follows T=20+70e−t/5T = 20 + 70e^{-t/5}. If the surrounding temperature changes with time, the difference D=T−AD = T - A obeys D′=−kD−A′(t)D' = -kD - A'(t), which is no longer pure decay.

Lessons Growth, decay and cooling

Logistic growth and harvesting

With growth rate r>0r > 0 and carrying capacity K>0K > 0, the logistic model and its solution for P(0)=P0>0P(0) = P_0 > 0 are

P′=rP(1−PK),P(t)=K1+Ae−rt,A=KP0−1.\begin{gathered} P' = rP\left(1 - \frac{P}{K}\right), \\[0.4em] P(t) = \frac{K}{1 + Ae^{-rt}}, \\[0.4em] A = \frac{K}{P_0} - 1. \end{gathered}

The equilibrium 00 is unstable and KK is asymptotically stable. The growth rate is largest at P=K/2P = K/2, where it equals rK/4rK/4. With a constant harvesting rate H≥0H \ge 0 the model becomes P′=rP(1−P/K)−HP' = rP(1 - P/K) - H.

Harvest rateEquilibriaBehavior
H<rK4H < \dfrac{rK}{4}P±=K2±K24−HKrP_\pm = \dfrac{K}{2} \pm \sqrt{\dfrac{K^2}{4} - \dfrac{HK}{r}}P−P_- is an unstable threshold and P+P_+ is stable
H=rK4H = \dfrac{rK}{4}K2\dfrac{K}{2} onlysemistable, populations below it decline
H>rK4H > \dfrac{rK}{4}nonePP decreases from every starting value

A population that starts below the threshold, or below K/2K/2 when H=rK/4H = rK/4, or at any level when H>rK/4H > rK/4, reaches 00 in finite time, where the model stops applying. With r=15r = \frac{1}{5} and K=100K = 100 the critical rate is 55, and H=165H = \frac{16}{5} gives the threshold 2020 and the stable level 8080.

Lessons Logistic growth and harvesting

Mixing and compartments

For a perfectly mixed tank holding an amount M(t)M(t) of a substance in volume V(t)V(t), with inflow rate qinq_{\text{in}} at concentration cinc_{\text{in}} and outflow rate qoutq_{\text{out}}, all constant,

M′=qin cin−qout MV,V(t)=V0+(qin−qout) t,\begin{gathered} M' = q_{\text{in}}\,c_{\text{in}} - q_{\text{out}}\,\frac{M}{V}, \\[0.4em] V(t) = V_0 + (q_{\text{in}} - q_{\text{out}})\,t, \end{gathered}

valid while the tank neither empties nor overflows. Solve for the amount, which is linear in MM, then compute the concentration c=M/Vc = M/V. For V=100+2tV = 100 + 2t liters with M(0)=100M(0) = 100 grams, an inflow of 33 liters per minute at 22 grams per liter and an outflow of 11 liter per minute, the equation M′=6−M/VM' = 6 - M/V has the integrating factor V\sqrt{V} and

M=2V−1000 V−1/2,c=2−1000 V−3/2.\begin{gathered} M = 2V - 1000\,V^{-1/2}, \\[0.4em] c = 2 - 1000\,V^{-3/2}. \end{gathered}

For several compartments, write one balance per compartment, and each exchange appears with opposite signs in the two compartments it connects. The system M1′=−(M1−M2)/10M_1' = -(M_1 - M_2)/10, M2′=(M1−M2)/10M_2' = (M_1 - M_2)/10 keeps M1+M2M_1 + M_2 constant, while the difference decays at rate 15\frac{1}{5}. With M1(0)=20M_1(0) = 20 and M2(0)=10M_2(0) = 10 this gives M1=15+5e−t/5M_1 = 15 + 5e^{-t/5} and M2=15−5e−t/5M_2 = 15 - 5e^{-t/5}.

Lessons Mixing and compartments

Drag and terminal velocity

Take velocity vv positive downward. A mass mm falling under gravity gg obeys mv′=mg+Fdragm v' = mg + F_{\text{drag}}, where the drag force always opposes the motion.

DragForceTerminal velocityVelocity
Linear−bv-bv, b>0b > 0vT=mgbv_T = \dfrac{mg}{b}vT+(v0−vT) e−bt/mv_T + (v_0 - v_T)\,e^{-bt/m} from v(0)=v0v(0) = v_0
Quadratic−c ∣v∣ v-c\,\lvert v \rvert\,v, c>0c > 0vT=mgcv_T = \sqrt{\dfrac{mg}{c}}vTtanh⁡gtvTv_T\tanh\dfrac{gt}{v_T} from rest

The terminal velocity is the equilibrium of the velocity equation, where gravity and drag balance. For quadratic drag the simpler form −cv2-cv^2 is valid only while v≥0v \ge 0, because an object moving upward has drag pointing down. With m=2m = 2, g=10g = 10 and b=4b = 4 in SI units, vT=5v_T = 5 meters per second and, from rest, v=5(1−e−2t)v = 5(1 - e^{-2t}). Quadratic drag with c=45c = \frac{4}{5} has the same terminal velocity but approaches it as v=5tanh⁡2tv = 5\tanh 2t.

Lessons Drag and terminal velocity

Second-Order and Higher-Order Equations

Linear equations have a clear solution structure. The solutions of a homogeneous equation of order nn form a family with exactly nn free constants, and every solution of a forced equation is one particular solution plus a homogeneous one. With constant coefficients the homogeneous solutions come from the roots of a polynomial.

Linear structure and the Wronskian

Write a linear equation of order nn in standard form, with leading coefficient 11,

L[y]=y(n)+pn−1(t) y(n−1)+⋯+p0(t) y=g(t).\begin{aligned} &L[y] = y^{(n)} + p_{n-1}(t)\,y^{(n-1)} \\[0.4em] &\quad + \cdots + p_0(t)\,y = g(t). \end{aligned}

A regular interval is an open interval II on which every pkp_k and gg is continuous. For every t0t_0 in II and any nn values y(t0),…,y(n−1)(t0)y(t_0), \ldots, y^{(n-1)}(t_0) there is exactly one solution, and it exists on all of II.

  • Superposition. Because LL is linear, sums and constant multiples of solutions of L[y]=0L[y] = 0 are again solutions. This fails for nonlinear equations and for L[y]=gL[y] = g with g≠0g \ne 0.
  • General solution. If y1,…,yny_1, \ldots, y_n is a fundamental set of solutions of L[y]=0L[y] = 0 on II and ypy_p is one solution of L[y]=gL[y] = g, every solution is c1y1+⋯+cnyn+ypc_1y_1 + \cdots + c_ny_n + y_p.
  • Wronskian. For two functions W=y1y2′−y1′y2W = y_1y_2' - y_1'y_2, and for nn functions it is the determinant of the matrix whose rows are the functions and their derivatives up to order n−1n - 1.

For solutions of y′′+p(t) y′+q(t) y=0y'' + p(t)\,y' + q(t)\,y = 0 on II, Abel’s identity

W(t)=W(t0) e−∫t0tp(s) dsW(t) = W(t_0)\,e^{-\int_{t_0}^{t} p(s)\,ds}

shows that WW is zero everywhere on II or nowhere, and order nn works the same way with pn−1p_{n-1} in place of pp. So nn solutions form a fundamental set exactly when W≠0W \ne 0 at one point of II. The test is about solutions. For arbitrary functions W=0W = 0 everywhere does not imply dependence, as t2t^2 and t∣t∣t\lvert t \rvert show on the whole line. For y′′−3y′+2y=0y'' - 3y' + 2y = 0 the solutions ete^t and e2te^{2t} have W=e3tW = e^{3t}, and y(0)=4y(0) = 4, y′(0)=5y'(0) = 5 select y=3et+e2ty = 3e^t + e^{2t}.

Lessons Superposition and Wronskians

Constant coefficients

For y′′+ay′+by=0y'' + ay' + by = 0 with real constants aa and bb, the function erte^{rt} is a solution exactly when r2+ar+b=0r^2 + ar + b = 0.

Roots of r2+ar+b=0r^2 + ar + b = 0General solution
real r1≠r2r_1 \ne r_2, when a2>4ba^2 > 4bC1er1t+C2er2tC_1e^{r_1t} + C_2e^{r_2t}
double root r=−a2r = -\dfrac{a}{2}, when a2=4ba^2 = 4b(C1+C2t) ert(C_1 + C_2t)\,e^{rt}
α±iβ\alpha \pm i\beta with β>0\beta > 0, when a2<4ba^2 < 4beαt(C1cos⁡βt+C2sin⁡βt)e^{\alpha t}(C_1\cos\beta t + C_2\sin\beta t)

For order nn, with characteristic polynomial anrn+⋯+a1r+a0a_nr^n + \cdots + a_1r + a_0 and an≠0a_n \ne 0, each real root rr of multiplicity mm contributes ert,tert,…,tm−1erte^{rt}, te^{rt}, \ldots, t^{m-1}e^{rt}, and each pair α±iβ\alpha \pm i\beta of multiplicity mm contributes tkeαtcos⁡βtt^ke^{\alpha t}\cos\beta t and tkeαtsin⁡βtt^ke^{\alpha t}\sin\beta t for k=0,…,m−1k = 0, \ldots, m - 1. The multiplicities add up to nn, so this gives a fundamental set of nn functions, and the nn initial values fix the constants.

  • The problem y′′+2y′+5y=0y'' + 2y' + 5y = 0, y(0)=3y(0) = 3, y′(0)=1y'(0) = 1 has roots −1±2i-1 \pm 2i and the solution e−t(3cos⁡2t+2sin⁡2t)e^{-t}(3\cos 2t + 2\sin 2t).
  • The polynomial (r2+1)2(r^2 + 1)^2 belongs to y(4)+2y′′+y=0y^{(4)} + 2y'' + y = 0, with the solutions cos⁡t\cos t, sin⁡t\sin t, tcos⁡tt\cos t and tsin⁡tt\sin t.
  • The polynomial (r−1)3(r - 1)^3 with y(0)=1y(0) = 1 and y′(0)=y′′(0)=0y'(0) = y''(0) = 0 gives y=et(1−t+12t2)y = e^t(1 - t + \frac{1}{2}t^2).

Lessons Second-order equations, Higher-order equations

Undetermined coefficients

The solution of L[y]=gL[y] = g is y=yc+ypy = y_c + y_p, a complementary solution ycy_c of L[y]=0L[y] = 0 plus one particular solution ypy_p. When LL has constant coefficients and gg is a sum of terms Pm(t) eαtcos⁡βtP_m(t)\,e^{\alpha t}\cos\beta t or Pm(t) eαtsin⁡βtP_m(t)\,e^{\alpha t}\sin\beta t, with PmP_m a polynomial of degree mm and possibly β=0\beta = 0, try for each term

yp=tseαt(A(t)cos⁡βt+B(t)sin⁡βt).\begin{aligned} y_p = t^s e^{\alpha t}\big(&A(t)\cos\beta t \\[0.4em] &+ B(t)\sin\beta t\big). \end{aligned}

Here AA and BB are polynomials of degree mm with unknown coefficients, both needed even when gg has only a cosine or only a sine, and ss is the multiplicity of α+iβ\alpha + i\beta as a root of the characteristic polynomial, with s=0s = 0 when it is not a root.

ForcingTrial when s=0s = 0
t2t^2At2+Bt+CAt^2 + Bt + C
e3te^{3t}Ae3tAe^{3t}
sin⁡ωt\sin\omega tAcos⁡ωt+Bsin⁡ωtA\cos\omega t + B\sin\omega t
te−tcos⁡2tte^{-t}\cos 2te−t((A1t+A0)cos⁡2t+(B1t+B0)sin⁡2t)e^{-t}\big((A_1t + A_0)\cos 2t + (B_1t + B_0)\sin 2t\big)

The factor tst^s handles overlap with the complementary solutions. For y′′−2y′+y=ety'' - 2y' + y = e^t the root 11 is double, so s=2s = 2, the trial is At2etAt^2e^t and A=12A = \frac{1}{2}. The annihilator method explains the rule. If a constant-coefficient operator Q(D)Q(D), with D=d/dtD = d/dt, satisfies Q(D) g=0Q(D)\,g = 0, then every solution of L[y]=gL[y] = g solves Q(D)L[y]=0Q(D)L[y] = 0, and the trial is a combination of the basis solutions of that larger equation that do not solve L[y]=0L[y] = 0. Here (D−1) et=0(D - 1)\,e^t = 0, so (D−1)3y=0(D - 1)^3y = 0 and the new term is t2ett^2e^t. For other forcings or variable coefficients, use variation of parameters.

Lessons Undetermined coefficients

Variation of parameters

Take y′′+p(t) y′+q(t) y=g(t)y'' + p(t)\,y' + q(t)\,y = g(t) in standard form, with pp, qq and gg continuous on II and a fundamental set y1y_1, y2y_2 with Wronskian WW. Then

yp=u1y1+u2y2,u1′=−y2 gW,u2′=y1 gW.\begin{gathered} y_p = u_1y_1 + u_2y_2, \\[0.4em] u_1' = -\frac{y_2\,g}{W}, \quad u_2' = \frac{y_1\,g}{W}. \end{gathered}

Taking u1u_1 and u2u_2 as integrals from t0t_0 to tt gives the particular solution with yp(t0)=yp′(t0)=0y_p(t_0) = y_p'(t_0) = 0. The leading coefficient must be 11, so for P y′′+Q y′+R y=GP\,y'' + Q\,y' + R\,y = G use g=G/Pg = G/P on an interval where P≠0P \ne 0. For y′′+y=sec⁡ty'' + y = \sec t on −π2<t<π2-\frac{\pi}{2} < t < \frac{\pi}{2}, with y1=cos⁡ty_1 = \cos t, y2=sin⁡ty_2 = \sin t and W=1W = 1, the derivatives are u1′=−tan⁡tu_1' = -\tan t and u2′=1u_2' = 1, and

yp=cos⁡t ln⁡(cos⁡t)+tsin⁡t.y_p = \cos t\,\ln(\cos t) + t\sin t.

Lessons Variation of parameters

Reduction of order and Cauchy-Euler equations

Reduction of order. If y1y_1 solves y′′+p y′+q y=0y'' + p\,y' + q\,y = 0 and y1≠0y_1 \ne 0 on an interval, then y=v y1y = v\,y_1 gives a first-order linear equation for v′v', and a second solution is

y2=y1∫e−∫p dty12 dt,y_2 = y_1\int \frac{e^{-\int p\,dt}}{y_1^2}\,dt,

with W(y1,y2)=e−∫p dt≠0W(y_1, y_2) = e^{-\int p\,dt} \ne 0. In the variable xx, the equation y′′−2xy′+2x2y=0y'' - \frac{2}{x}y' + \frac{2}{x^2}y = 0 on x>0x > 0 with y1=xy_1 = x gives y2=x2y_2 = x^2.

Cauchy-Euler equations. In the variable xx, the equation x2y′′+axy′+by=0x^2y'' + axy' + by = 0 is studied on x>0x > 0. The trial y=xmy = x^m gives the indicial equation m(m−1)+am+b=0m(m - 1) + am + b = 0. Equivalently, x=eτx = e^\tau turns it into Y′′+(a−1) Y′+bY=0Y'' + (a - 1)\,Y' + bY = 0 with constant coefficients, where Y(τ)=y(eτ)Y(\tau) = y(e^\tau) and primes on YY mean d/dτd/d\tau.

RootsSolutions on x>0x > 0
real m1≠m2m_1 \ne m_2xm1x^{m_1} and xm2x^{m_2}
double root mmxmx^m and xmln⁡xx^m\ln x
α±iβ\alpha \pm i\beta with β>0\beta > 0xαcos⁡(βln⁡x)x^\alpha\cos(\beta\ln x) and xαsin⁡(βln⁡x)x^\alpha\sin(\beta\ln x)

On x<0x < 0 replace xx by ∣x∣\lvert x \rvert in the solutions. The origin is a singular point where initial data do not select one solution, since y=0y = 0 and y=x2y = x^2 both solve x2y′′−2y=0x^2y'' - 2y = 0 with y(0)=y′(0)=0y(0) = y'(0) = 0. For x2y′′−3xy′+4y=0x^2y'' - 3xy' + 4y = 0 the indicial equation is (m−2)2=0(m - 2)^2 = 0, and y(1)=1y(1) = 1, y′(1)=0y'(1) = 0 give y=x2(1−2ln⁡x)y = x^2(1 - 2\ln x).

Lessons Reduction of order, Cauchy-Euler equations

Nonlinear equations that lose an order

In the variable xx, two kinds of second-order equation reduce to first order.

Missing variableFormSubstitutionResult
dependent variable yyy′′=F(x,y′)y'' = F(x, y')v=y′v = y'v′=F(x,v)v' = F(x, v)
independent variable xxy′′=G(y,y′)y'' = G(y, y')v=y′v = y' as a function of yyv dvdy=G(y,v)v\,\dfrac{dv}{dy} = G(y, v)

The second substitution uses yy as the coordinate, so it is valid only on an interval where y′≠0y' \ne 0. After finding vv, solve y′=vy' = v for yy. For y′′=(y′)2y'' = (y')^2 with y(0)=2y(0) = 2 and y′(0)=1y'(0) = 1, the first substitution gives v=11−xv = \frac{1}{1 - x} and y=2−ln⁡(1−x)y = 2 - \ln(1 - x) on x<1x < 1. For y′′=2y3y'' = 2y^3 with y(0)=y′(0)=1y(0) = y'(0) = 1, the second gives the first integral (y′)2−y4=0(y')^2 - y^4 = 0, the data choose y′=y2y' = y^2, and y=11−xy = \frac{1}{1 - x} on x<1x < 1.

Lessons Order reduction in nonlinear equations

Oscillations, Response and Circuits

A mass on a spring and a series electric circuit obey the same second-order linear equation. Damping decides how free motion dies out, the forcing frequency decides how large the response is, and complex arithmetic gives the amplitude and phase of a steady response in one step.

Free and damped oscillations

A mass m>0m > 0 on a spring of stiffness k>0k > 0 with viscous damping c≥0c \ge 0, whose displacement from equilibrium is x(t)x(t), satisfies mx′′+cx′+kx=0m x'' + c x' + k x = 0. The natural angular frequency is ω0=k/m\omega_0 = \sqrt{k/m}, and the characteristic roots are

r=−c±c2−4mk2m.r = \frac{-c \pm \sqrt{c^2 - 4mk}}{2m}.
CaseConditionSolution
Undampedc=0c = 0C1cos⁡ω0t+C2sin⁡ω0tC_1\cos\omega_0t + C_2\sin\omega_0t
Underdamped0<c2<4mk0 < c^2 < 4mke−ct/(2m)(C1cos⁡ωdt+C2sin⁡ωdt)e^{-ct/(2m)}(C_1\cos\omega_dt + C_2\sin\omega_dt)
Critically dampedc2=4mkc^2 = 4mk(C1+C2t) e−ct/(2m)(C_1 + C_2t)\,e^{-ct/(2m)}
Overdampedc2>4mkc^2 > 4mkC1er1t+C2er2tC_1e^{r_1t} + C_2e^{r_2t} with r2<r1<0r_2 < r_1 < 0

Here ωd=ω02−c2/(4m2)\omega_d = \sqrt{\omega_0^2 - c^2/(4m^2)} is the damped angular frequency. With c>0c > 0 every solution tends to 00, and critically damped or overdamped motion crosses equilibrium at most once. The mechanical energy

E=12m (x′)2+12k x2E = \tfrac{1}{2}m\,(x')^2 + \tfrac{1}{2}k\,x^2

satisfies E′=−c (x′)2≤0E' = -c\,(x')^2 \le 0, so damping removes energy. With m=1m = 1, c=2c = 2 and k=5k = 5 in SI units, the release x(0)=2x(0) = 2, x′(0)=0x'(0) = 0 gives x=e−t(2cos⁡2t+sin⁡2t)x = e^{-t}(2\cos 2t + \sin 2t), with ωd=2\omega_d = 2 and starting energy 1010 joules.

Lessons Free and damped oscillations

Forcing, beats and resonance

For mx′′+cx′+kx=F0cos⁡ωtm x'' + c x' + k x = F_0\cos\omega t with F0≠0F_0 \ne 0, three cases matter.

  • No damping, ω≠ω0\omega \ne \omega_0. A particular solution is F0m(ω02−ω2)cos⁡ωt\frac{F_0}{m(\omega_0^2 - \omega^2)}\cos\omega t. Starting from rest, the motion is that amplitude times cos⁡ωt−cos⁡ω0t\cos\omega t - \cos\omega_0t, and for ω\omega near ω0\omega_0 it shows beats, a fast oscillation inside a slow envelope that repeats with the beat period 2π/∣ω0−ω∣2\pi/\lvert\omega_0 - \omega\rvert.
  • No damping, ω=ω0\omega = \omega_0. This is resonance. A particular solution is F02mω0 tsin⁡ω0t\frac{F_0}{2m\omega_0}\,t\sin\omega_0t, which grows linearly without bound.
  • Damping, c>0c > 0. Every solution approaches a steady periodic response of amplitude
A(ω)=F0(k−mω2)2+c2ω2.A(\omega) = \frac{F_0}{\sqrt{(k - m\omega^2)^2 + c^2\omega^2}}.

The amplitude has a peak at a positive frequency only when c2<2mkc^2 < 2mk, a stronger condition than underdamping. The peak is at

ωpeak=km−c22m2,\omega_{\text{peak}} = \sqrt{\frac{k}{m} - \frac{c^2}{2m^2}},

with peak amplitude F0c ωd\frac{F_0}{c\,\omega_d}, and when c2≥2mkc^2 \ge 2mk the amplitude decreases for all ω>0\omega > 0. As examples, x′′+9x=5cos⁡2tx'' + 9x = 5\cos 2t from rest gives x=cos⁡2t−cos⁡3t=2sin⁡t2sin⁡5t2x = \cos 2t - \cos 3t = 2\sin\frac{t}{2}\sin\frac{5t}{2}, and x′′+9x=6cos⁡3tx'' + 9x = 6\cos 3t has the resonant solution x=tsin⁡3tx = t\sin 3t.

Lessons Forced oscillations and resonance

Steady state and frequency response

For c>0c > 0 and the input F0cos⁡ωtF_0\cos\omega t, solve the complex equation with input F0eiωtF_0e^{i\omega t} instead and take the real part. The complex gain, or frequency response, is

H(iω)=1k−mω2+icω,H(i\omega) = \frac{1}{k - m\omega^2 + ic\omega},

and the steady response is

xss=Acos⁡(ωt−φ),A=F0 ∣H(iω)∣.\begin{gathered} x_{\text{ss}} = A\cos(\omega t - \varphi), \\[0.4em] A = F_0\,\lvert H(i\omega) \rvert. \end{gathered}

The amplitude ratio is ∣H(iω)∣\lvert H(i\omega) \rvert, and the phase lag φ=−arg⁡H(iω)\varphi = -\arg H(i\omega) is the angle of the point (k−mω2,cω)(k - m\omega^2, c\omega), which lies between 00 and π\pi for ω>0\omega > 0. Compute it with the two-argument arctangent, because arctan⁡cωk−mω2\arctan\frac{c\omega}{k - m\omega^2} gives the wrong quadrant when mω2>km\omega^2 > k. The full solution is xssx_{\text{ss}} plus a transient, a homogeneous solution fitted to the initial values, and the transient decays because c>0c > 0.

For x′′+2x′+5x=13cos⁡2tx'' + 2x' + 5x = 13\cos 2t the gain is H(2i)=11+4iH(2i) = \frac{1}{1 + 4i}, so the amplitude is 1317≈3.15\frac{13}{\sqrt{17}} \approx 3.15, the phase lag is arctan⁡4≈1.33\arctan 4 \approx 1.33 radians, and xss=1317cos⁡2t+5217sin⁡2tx_{\text{ss}} = \frac{13}{17}\cos 2t + \frac{52}{17}\sin 2t. From rest the transient is −e−t(1317cos⁡2t+11734sin⁡2t)-e^{-t}\big(\frac{13}{17}\cos 2t + \frac{117}{34}\sin 2t\big).

Lessons Sinusoidal steady state

RLC circuits

In a series circuit with inductance LL, resistance RR, capacitance CC and applied voltage V(t)V(t), let qq be the charge on the capacitor and i=q′i = q' the current. Kirchhoff’s voltage law adds the voltage drops L i′L\,i', R iR\,i and q/Cq/C, which gives

L q′′+R q′+qC=V(t).L\,q'' + R\,q' + \frac{q}{C} = V(t).
MechanicalElectrical
mass mminductance LL
damping ccresistance RR
stiffness kk1/C1/C
force F(t)F(t)voltage V(t)V(t)
displacement xxcharge qq
velocity x′x'current ii

So ω0=1/LC\omega_0 = 1/\sqrt{LC}, and a free circuit is underdamped when 0<R2<4L/C0 < R^2 < 4L/C. The stored energy E=12L i2+q22CE = \frac{1}{2}L\,i^2 + \frac{q^2}{2C} satisfies E′=V i−R i2E' = V\,i - R\,i^2. With L=1L = 1, R=2R = 2, C=15C = \frac{1}{5} in SI units, V=0V = 0, q(0)=1q(0) = 1 and i(0)=0i(0) = 0, the charge is q=e−t(cos⁡2t+12sin⁡2t)q = e^{-t}(\cos 2t + \frac{1}{2}\sin 2t), the current is i=−52e−tsin⁡2ti = -\frac{5}{2}e^{-t}\sin 2t and the starting energy is 52\frac{5}{2} joules.

Lessons RLC circuits

Laplace Transforms and Causal Response

The Laplace transform turns a linear initial value problem with constant coefficients into algebra in a new variable ss, with the initial values built in. It handles switched inputs, impulses and convolutions directly, and partial fractions invert the rational functions it produces. Time starts at t=0t = 0, and functions are considered for t≥0t \ge 0 only.

Definition and existence

For ff defined for t≥0t \ge 0, the one-sided Laplace transform is

F(s)=L{f}(s)=∫0∞e−stf(t) dt\begin{aligned} F(s) &= \mathcal{L}\{f\}(s) \\[0.4em] &= \int_0^\infty e^{-st}f(t)\,dt \end{aligned}

for the ss where the improper integral converges. A function is of exponential order aa when ∣f(t)∣≤Meat\lvert f(t) \rvert \le Me^{at} for all t≥T0t \ge T_0, for some constants MM and T0T_0.

  • Existence. If ff is piecewise continuous on every interval [0,T][0, T] and of exponential order aa, the integral converges absolutely for Re⁡s>a\operatorname{Re} s > a, or s>as > a for real ss, and F(s)→0F(s) \to 0 as real s→∞s \to \infty. The actual region of convergence can be larger, and some functions such as et2e^{t^2} have no Laplace transform at all.
  • Linearity. The Laplace transform of αf+βg\alpha f + \beta g is αF+βG\alpha F + \beta G on the region where both Laplace transforms exist.
  • Uniqueness. Two piecewise continuous functions of exponential order with the same Laplace transform agree at every t>0t > 0 where both are continuous. The inverse Laplace transform is therefore unique up to the values at isolated points such as jumps.

For example f=2+t+e3tf = 2 + t + e^{3t} has F(s)=2s+1s2+1s−3F(s) = \frac{2}{s} + \frac{1}{s^2} + \frac{1}{s - 3} for Re⁡s>3\operatorname{Re} s > 3.

Lessons The Laplace transform

Table of Laplace transforms

Here aa, bb and cc are real constants and u(t−c)u(t - c) is the unit step at cc.

f(t)f(t)F(s)F(s)Region
111s\dfrac{1}{s}Re⁡s>0\operatorname{Re} s > 0
tnt^n, n=0,1,2,…n = 0, 1, 2, \ldotsn!sn+1\dfrac{n!}{s^{n+1}}Re⁡s>0\operatorname{Re} s > 0
eate^{at}1s−a\dfrac{1}{s - a}Re⁡s>a\operatorname{Re} s > a
tneatt^ne^{at}n!(s−a)n+1\dfrac{n!}{(s - a)^{n+1}}Re⁡s>a\operatorname{Re} s > a
sin⁡bt\sin btbs2+b2\dfrac{b}{s^2 + b^2}Re⁡s>0\operatorname{Re} s > 0
cos⁡bt\cos btss2+b2\dfrac{s}{s^2 + b^2}Re⁡s>0\operatorname{Re} s > 0
eatsin⁡bte^{at}\sin btb(s−a)2+b2\dfrac{b}{(s - a)^2 + b^2}Re⁡s>a\operatorname{Re} s > a
eatcos⁡bte^{at}\cos bts−a(s−a)2+b2\dfrac{s - a}{(s - a)^2 + b^2}Re⁡s>a\operatorname{Re} s > a
sinh⁡bt\sinh btbs2−b2\dfrac{b}{s^2 - b^2}Re⁡s>∣b∣\operatorname{Re} s > \lvert b \rvert
cosh⁡bt\cosh btss2−b2\dfrac{s}{s^2 - b^2}Re⁡s>∣b∣\operatorname{Re} s > \lvert b \rvert
tsin⁡btt\sin bt2bs(s2+b2)2\dfrac{2bs}{(s^2 + b^2)^2}Re⁡s>0\operatorname{Re} s > 0
u(t−c)u(t - c), c≥0c \ge 0e−css\dfrac{e^{-cs}}{s}Re⁡s>0\operatorname{Re} s > 0
δ(t−c)\delta(t - c), c>0c > 0e−cse^{-cs}all ss
Rulef(t)f(t)F(s)F(s)
First shifteatf(t)e^{at}f(t)F(s−a)F(s - a)
Second shift, c≥0c \ge 0u(t−c) f(t−c)u(t - c)\,f(t - c)e−csF(s)e^{-cs}F(s)
Derivativef′(t)f'(t)sF(s)−f(0)sF(s) - f(0)
Second derivativef′′(t)f''(t)s2F(s)−sf(0)−f′(0)s^2F(s) - sf(0) - f'(0)
Multiplication by ttt f(t)t\,f(t)−F′(s)-F'(s)
Integral∫0tf(τ) dτ\displaystyle\int_0^t f(\tau)\,d\tauF(s)s\dfrac{F(s)}{s}
Convolution(f∗g)(t)(f * g)(t)F(s) G(s)F(s)\,G(s)
Period T>0T > 0f(t+T)=f(t)f(t + T) = f(t)11−e−sT∫0Te−stf(t) dt\dfrac{1}{1 - e^{-sT}}\displaystyle\int_0^T e^{-st}f(t)\,dt

Each rule holds where the Laplace transforms involved exist. The derivative rule needs ff continuous on [0,∞)[0, \infty) and of exponential order aa with f′f' piecewise continuous, and then it holds for Re⁡s>a\operatorname{Re} s > a. The second derivative rule needs the same of ff and f′f', with f′′f'' piecewise continuous. The periodic formula holds for Re⁡s>0\operatorname{Re} s > 0 when ff is piecewise continuous, and the convolution is defined in the last topic of this unit.

Lessons The Laplace transform, Derivatives, Shifts, Convolution

Inverting by partial fractions

To find ff from a rational F=P/QF = P/Q with deg⁡P<deg⁡Q\deg P < \deg Q, factor QQ over the real numbers and split FF into partial fractions.

Factor of QQTermsInverse Laplace transform
s−as - aAs−a\dfrac{A}{s - a}AeatAe^{at}
(s−a)k(s - a)^k, a repeated poleA1s−a+⋯+Ak(s−a)k\dfrac{A_1}{s - a} + \cdots + \dfrac{A_k}{(s - a)^k}∑j=1kAj tj−1(j−1)! eat\displaystyle\sum_{j=1}^{k}\frac{A_j\,t^{j-1}}{(j - 1)!}\,e^{at}
(s+α)2+β2(s + \alpha)^2 + \beta^2 with β>0\beta > 0C(s+α)+Dβ(s+α)2+β2\dfrac{C(s + \alpha) + D\beta}{(s + \alpha)^2 + \beta^2}e−αt(Ccos⁡βt+Dsin⁡βt)e^{-\alpha t}(C\cos\beta t + D\sin\beta t)

An irreducible quadratic is put in this form by completing the square, as in s2+4s+13=(s+2)2+9s^2 + 4s + 13 = (s + 2)^2 + 9. For example

2s+5s2+2s+5=2(s+1)+32⋅2(s+1)2+4,\begin{aligned} &\frac{2s + 5}{s^2 + 2s + 5} \\[0.4em] &\quad = \frac{2(s + 1) + \frac{3}{2}\cdot 2}{(s + 1)^2 + 4}, \end{aligned}

so its inverse is e−t(2cos⁡2t+32sin⁡2t)e^{-t}\big(2\cos 2t + \frac{3}{2}\sin 2t\big). Similarly 1s2(s+1)=−1s+1s2+1s+1\frac{1}{s^2(s + 1)} = -\frac{1}{s} + \frac{1}{s^2} + \frac{1}{s + 1} has the inverse t−1+e−tt - 1 + e^{-t}. When f′f' is also piecewise continuous and of exponential order, the initial value theorem f(0+)=lim⁡s→∞sF(s)f(0^+) = \lim_{s\to\infty} sF(s) gives a quick check, here f(0)=2f(0) = 2 for the first example.

Lessons Inverse Laplace transforms

Solving initial value problems

Take the Laplace transform of the equation, use the derivative rules so that the initial values appear, solve the resulting algebraic equation for Y(s)Y(s) and invert. For ay′′+by′+cy=g(t)ay'' + by' + cy = g(t) with y(0)=y0y(0) = y_0 and y′(0)=y1y'(0) = y_1,

(as2+bs+c) Y(s)=G(s)+a(sy0+y1)+by0.\begin{gathered} (as^2 + bs + c)\,Y(s) = G(s) \\[0.4em] {} + a(sy_0 + y_1) + by_0. \end{gathered}

For y′′+3y′+2y=4y'' + 3y' + 2y = 4 with y(0)=1y(0) = 1 and y′(0)=0y'(0) = 0,

Y(s)=s2+3s+4s(s+1)(s+2)=2s−2s+1+1s+2,\begin{aligned} Y(s) &= \frac{s^2 + 3s + 4}{s(s + 1)(s + 2)} \\[0.4em] &= \frac{2}{s} - \frac{2}{s + 1} + \frac{1}{s + 2}, \end{aligned}

so y=2−2e−t+e−2ty = 2 - 2e^{-t} + e^{-2t}. The zeros of as2+bs+cas^2 + bs + c are the characteristic roots, and apart from the poles of GG they are the only possible poles of YY.

Lessons Initial value problems

Step functions and switched inputs

The unit step u(t−c)u(t - c) is 00 for t<ct < c and 11 for t>ct > c, and its value at cc does not affect any Laplace transform. An input equal to g1(t)g_1(t) for 0≤t<c0 \le t < c and to g2(t)g_2(t) for t≥ct \ge c is g1(t)+u(t−c)(g2(t)−g1(t))g_1(t) + u(t - c)\big(g_2(t) - g_1(t)\big). The second shift theorem needs the switched part written in the elapsed time t−ct - c, which gives

L{u(t−c) g(t)}=e−cs L{g(t+c)},\begin{aligned} &\mathcal{L}\{u(t - c)\,g(t)\} \\[0.4em] &\quad = e^{-cs}\,\mathcal{L}\{g(t + c)\}, \end{aligned}

where the last Laplace transform is taken of the function t↦g(t+c)t \mapsto g(t + c). So u(t−2) (t−2)u(t - 2)\,(t - 2) has the Laplace transform e−2ss2\frac{e^{-2s}}{s^2}, while u(t−2) tu(t - 2)\,t, which jumps to height 22, has e−2s(1s2+2s)e^{-2s}\big(\frac{1}{s^2} + \frac{2}{s}\big). For y′+y=u(t−2) (t−2)y' + y = u(t - 2)\,(t - 2) with y(0)=0y(0) = 0, the Laplace transform Y=e−2ss2(s+1)Y = \frac{e^{-2s}}{s^2(s + 1)} gives y=u(t−2)(t−3+e−(t−2))y = u(t - 2)\big(t - 3 + e^{-(t - 2)}\big), that is,

y={0,t<2,t−3+e2−t,t≥2.y = \begin{cases} 0, & t < 2, \\ t - 3 + e^{2 - t}, & t \ge 2. \end{cases}

The pulse of period 22 that equals 11 on [0,1)[0, 1) and 00 on [1,2)[1, 2) has, by the periodic formula, the Laplace transform 1s(1+e−s)\frac{1}{s(1 + e^{-s})} for Re⁡s>0\operatorname{Re} s > 0.

Lessons Step functions and periodic inputs

Convolution, impulses and transfer functions

The causal convolution of two functions on t≥0t \ge 0 is

(f∗g)(t)=∫0tf(t−τ) g(τ) dτ,\begin{aligned} &(f * g)(t) \\[0.4em] &\quad = \int_0^t f(t - \tau)\,g(\tau)\,d\tau, \end{aligned}

and f∗g=g∗ff * g = g * f. For piecewise continuous ff and gg of exponential order, L{f∗g}=F(s) G(s)\mathcal{L}\{f * g\} = F(s)\,G(s), while the Laplace transform of the product fgfg is not FGFG in general.

For any(n)+⋯+a0y=ga_ny^{(n)} + \cdots + a_0y = g with all initial values 00, the Laplace transform gives Y(s)=H(s) G(s)Y(s) = H(s)\,G(s) with the transfer function

H(s)=1ansn+⋯+a0.H(s) = \frac{1}{a_ns^n + \cdots + a_0}.

The impulse response hh, the inverse Laplace transform of HH, gives the zero-state response y=h∗gy = h * g. With nonzero initial values, add the zero-input response, the solution of the homogeneous equation with those values.

The Dirac delta δ(t−t0)\delta(t - t_0) is a distribution, not a function. It acts on a continuous function φ\varphi by ∫δ(t−t0) φ(t) dt=φ(t0)\int \delta(t - t_0)\,\varphi(t)\,dt = \varphi(t_0) and is the limit of pulses of area 11 that shrink to the point t0t_0. In ay′′+by′+cy=J δ(t−t0)ay'' + by' + cy = J\,\delta(t - t_0) with t0>0t_0 > 0, the solution yy stays continuous at t0t_0 and y′y' jumps by J/aJ/a.

The poles of HH are the zeros of its denominator after cancelling common factors. A rational HH whose numerator degree does not exceed its denominator degree is bounded-input bounded-output stable exactly when every pole has negative real part. A pole on the imaginary axis is enough to lose this, as in undamped resonance. The poles describe the input to output behavior with zero initial state, and a cancelled factor can hide an internal mode that grows from nonzero initial values.

  • For y′′+3y′+2y=3 δ(t−2)y'' + 3y' + 2y = 3\,\delta(t - 2) from rest, H=1(s+1)(s+2)H = \frac{1}{(s + 1)(s + 2)} and h=e−t−e−2th = e^{-t} - e^{-2t} for t≥0t \ge 0, so y=3u(t−2)(e−(t−2)−e−2(t−2))y = 3u(t - 2)\big(e^{-(t - 2)} - e^{-2(t - 2)}\big). The poles −1-1 and −2-2 give a stable system, with ∫0∞∣h∣ dt=12\int_0^\infty \lvert h \rvert\,dt = \frac{1}{2}.
  • For y′+2y=3y' + 2y = 3 with y(0)=0y(0) = 0, the convolution gives y=∫0t3e−2(t−τ) dτ=32(1−e−2t)y = \int_0^t 3e^{-2(t - \tau)}\,d\tau = \frac{3}{2}(1 - e^{-2t}), which also solves the Volterra integral equation y(t)=3t−2∫0ty(τ) dτy(t) = 3t - 2\int_0^t y(\tau)\,d\tau.

Lessons Convolution, Impulses and transfer functions

Series Solutions and Special Functions

When the coefficients vary, a solution can be built as a power series whose coefficients follow a recurrence. Where the normalized coefficients are analytic this always works, and at a regular singular point a leading power xrx^r, sometimes with a logarithm, is needed. The Bessel and Legendre functions arise this way. In this unit the independent variable is xx and primes mean d/dxd/dx.

Power series at an ordinary point

Divide by the leading coefficient to get y′′+p(x) y′+q(x) y=0y'' + p(x)\,y' + q(x)\,y = 0. A point x0x_0 is ordinary when pp and qq are analytic there, meaning they equal convergent power series in x−x0x - x_0, and singular otherwise.

  1. Substitute y=∑an(x−x0)ny = \sum a_n(x - x_0)^n and the series for y′y' and y′′y''.
  2. Shift indices so that every sum shows the same power (x−x0)n(x - x_0)^n.
  3. Set each coefficient to 00 to get a recurrence. The first two coefficients a0=y(x0)a_0 = y(x_0) and a1=y′(x0)a_1 = y'(x_0) are free and determine the rest.

For y′′+2xy′=0y'' + 2xy' = 0 at 00, matching powers gives (n+2)(n+1) an+2+2n an=0(n + 2)(n + 1)\,a_{n+2} + 2n\,a_n = 0, so for n≥0n \ge 0

an+2=−2n an(n+2)(n+1).a_{n+2} = -\frac{2n\,a_n}{(n + 2)(n + 1)}.

The even and odd coefficients form separate chains. The choice a0=1a_0 = 1, a1=0a_1 = 0 gives y=1y = 1, and a0=0a_0 = 0, a1=1a_1 = 1 gives

y=x−x33+x510−⋯ ,y = x - \frac{x^3}{3} + \frac{x^5}{10} - \cdots,

which is ∫0xe−u2 du\int_0^x e^{-u^2}\,du.

Lessons Power-series solutions

Radius of convergence

If pp and qq are analytic in the disk ∣x−x0∣<ρ\lvert x - x_0 \rvert < \rho of the complex plane, the power series about x0x_0 of every solution converges at least for ∣x−x0∣<ρ\lvert x - x_0 \rvert < \rho. For polynomial coefficients P2(x) y′′+P1(x) y′+P0(x) y=0P_2(x)\,y'' + P_1(x)\,y' + P_0(x)\,y = 0 with P2(x0)≠0P_2(x_0) \ne 0, this guaranteed radius is at least the distance from x0x_0 to the nearest complex zero of P2P_2. Complex zeros count, so for (1+x2) y′′+y=0(1 + x^2)\,y'' + y = 0 at 00 the zeros ±i\pm i give a radius of at least 11, although the coefficients are smooth on the whole real line.

This is a lower bound, not the exact radius. The equation (1−x) y′′−2y′=0(1 - x)\,y'' - 2y' = 0 has the singular point 11 and the solutions C0+C11−xC_0 + \frac{C_1}{1 - x}. The solution 11 has an infinite radius, while 11−x=∑xn\frac{1}{1 - x} = \sum x^n has radius exactly 11. A series also diverges outside its radius where the function is still defined, since ∑xn\sum x^n diverges at x=−1x = -1, where 11−x=12\frac{1}{1 - x} = \frac{1}{2}.

Lessons Analyticity and convergence

Regular singular points and Frobenius series

A singular point x0=0x_0 = 0 is regular when x p(x)x\,p(x) and x2q(x)x^2q(x) are analytic at 00. Then the equation can be written as x2y′′+x P(x) y′+Q(x) y=0x^2y'' + x\,P(x)\,y' + Q(x)\,y = 0 with P=xpP = xp and Q=x2qQ = x^2q analytic, and on 0<x<ρ0 < x < \rho one looks for Frobenius series

y=xr∑n=0∞anxn,a0≠0.y = x^r\sum_{n=0}^{\infty} a_nx^n, \quad a_0 \ne 0.

The lowest power gives the indicial equation r(r−1)+P(0) r+Q(0)=0r(r - 1) + P(0)\,r + Q(0) = 0. For real roots r1≥r2r_1 \ge r_2,

RootsTwo independent solutions
r1−r2r_1 - r_2 not an integera Frobenius series for each root
r1=r2r_1 = r_2y1y_1 for r1r_1, and y2=y1ln⁡x+xr1∑n≥1bnxny_2 = y_1\ln x + x^{r_1}\sum_{n \ge 1} b_nx^n
r1−r2r_1 - r_2 a positive integery1y_1 for r1r_1, and y2=C y1ln⁡x+xr2∑n≥0bnxny_2 = C\,y_1\ln x + x^{r_2}\sum_{n \ge 0} b_nx^n with b0≠0b_0 \ne 0, where CC may be 00

The series converge at least on 0<x<ρ0 < x < \rho, where ρ\rho is the smaller radius of convergence of PP and QQ. Complex roots α±iβ\alpha \pm i\beta give complex solutions whose real and imaginary parts are real solutions, such as xαcos⁡(βln⁡x)x^\alpha\cos(\beta\ln x) in the simplest case. The equation x2y′′+xy′−49y=0x^2y'' + xy' - \frac{4}{9}y = 0 has the roots ±23\pm\frac{2}{3} and the solutions x2/3x^{2/3} and x−2/3x^{-2/3}. With 14\frac{1}{4} in place of 49\frac{4}{9} the roots ±12\pm\frac{1}{2} differ by an integer, yet x1/2x^{1/2} and x−1/2x^{-1/2} need no logarithm. The equation x2y′′+xy′=0x^2y'' + xy' = 0 has the double root 00 and the solutions 11 and ln⁡x\ln x.

Lessons Regular singular points and Frobenius

Bessel functions

Bessel’s equation of order ν≥0\nu \ge 0, for x>0x > 0, is

x2y′′+xy′+(x2−ν2) y=0,\begin{aligned} &x^2y'' + xy' \\[0.4em] &\quad + (x^2 - \nu^2)\,y = 0, \end{aligned}

with indicial roots ±ν\pm\nu. The Bessel function of the first kind is

Jν(x)=∑m=0∞cm(x2)2m+ν,cm=(−1)mm! Γ(m+ν+1).\begin{gathered} J_\nu(x) = \sum_{m=0}^{\infty} c_m\left(\frac{x}{2}\right)^{2m + \nu}, \\[0.4em] c_m = \frac{(-1)^m}{m!\,\Gamma(m + \nu + 1)}. \end{gathered}

So J0(x)=1−x24+x464−x62304+⋯J_0(x) = 1 - \frac{x^2}{4} + \frac{x^4}{64} - \frac{x^6}{2304} + \cdots, with J0(0)=1J_0(0) = 1. The general solution on x>0x > 0 is C1Jν+C2YνC_1J_\nu + C_2Y_\nu, where the Bessel function of the second kind YνY_\nu is unbounded as x→0+x \to 0^+, for example Y0(x)≈2πln⁡xY_0(x) \approx \frac{2}{\pi}\ln x there. When ν\nu is not an integer, J−νJ_{-\nu} can be used in place of YνY_\nu. The half-order functions are elementary,

J1/2(x)=2πx sin⁡x,J−1/2(x)=2πx cos⁡x.\begin{gathered} J_{1/2}(x) = \sqrt{\frac{2}{\pi x}}\,\sin x, \\[0.4em] J_{-1/2}(x) = \sqrt{\frac{2}{\pi x}}\,\cos x. \end{gathered}

In radial problems the equation R′′+1rR′+(κ2−m2r2)R=0R'' + \frac{1}{r}R' + \big(\kappa^2 - \frac{m^2}{r^2}\big)R = 0, with primes meaning d/drd/dr, becomes Bessel’s equation of order mm in x=κrx = \kappa r. The solutions that stay bounded at r=0r = 0 are the multiples of Jm(κr)J_m(\kappa r).

Lessons Bessel functions

Legendre polynomials

Legendre’s equation on −1<x<1-1 < x < 1, for a constant n≥0n \ge 0, is

(1−x2) y′′−2xy′+n(n+1) y=0.\begin{aligned} &(1 - x^2)\,y'' - 2xy' \\[0.4em] &\quad + n(n + 1)\,y = 0. \end{aligned}

The point 00 is ordinary and the series coefficients satisfy

ak+2=(k−n)(k+n+1)(k+2)(k+1) ak.\begin{aligned} &a_{k+2} \\[0.4em] &\quad = \frac{(k - n)(k + n + 1)}{(k + 2)(k + 1)}\,a_k. \end{aligned}

When nn is a nonnegative integer, the chain with the parity of nn stops at k=nk = n, which gives a polynomial. The Legendre polynomial PnP_n is that polynomial scaled so that Pn(1)=1P_n(1) = 1, and Rodrigues’ formula gives it directly as Pn(x)=12nn!dndxn(x2−1)nP_n(x) = \frac{1}{2^nn!}\frac{d^n}{dx^n}(x^2 - 1)^n.

nn00112233
Pn(x)P_n(x)11xx3x2−12\dfrac{3x^2 - 1}{2}5x3−3x2\dfrac{5x^3 - 3x}{2}

The polynomials are orthogonal on [−1,1][-1, 1],

∫−11PmPn dx={0,m≠n,22n+1,m=n.\int_{-1}^{1} P_mP_n\,dx = \begin{cases} 0, & m \ne n, \\ \dfrac{2}{2n + 1}, & m = n. \end{cases}

In the form ((1−x2) y′)′+n(n+1) y=0\big((1 - x^2)\,y'\big)' + n(n + 1)\,y = 0, the polynomials PnP_n are the eigenfunctions of a singular Sturm-Liouville problem with eigenvalues n(n+1)n(n + 1), where boundedness at x=±1x = \pm 1 takes the place of boundary conditions. For integer nn every solution that is not a multiple of PnP_n is unbounded near x=1x = 1 or x=−1x = -1.

Lessons Legendre polynomials

Linear Systems and Normal Modes

Several interacting quantities form a first-order system for a state vector. For a constant matrix, eigenvalues and eigenvectors give the solutions, the matrix exponential packages them into one map, and in the plane the eigenvalues sort the phase portraits. Undamped coupled oscillators separate into independent normal modes.

Systems and conversion

A first-order system for the state X(t)=(x1,…,xn)TX(t) = (x_1, \ldots, x_n)^T is X′=F(t,X)X' = F(t, X), autonomous when FF does not depend on tt. It is linear when X′=A(t) X+b(t)X' = A(t)\,X + b(t), and if A(t)A(t) and b(t)b(t) are continuous on an open interval II, each initial value X(t0)=X0X(t_0) = X_0 with t0t_0 in II gives exactly one solution, defined on all of II.

An equation y(n)=G(t,y,y′,…,y(n−1))y^{(n)} = G(t, y, y', \ldots, y^{(n-1)}) becomes a system with X=(y,y′,…,y(n−1))TX = (y, y', \ldots, y^{(n-1)})^T, so that x1′=x2x_1' = x_2, up to xn−1′=xnx_{n-1}' = x_n, and xn′=Gx_n' = G. The initial data become the vector X(t0)=(y(t0),…,y(n−1)(t0))TX(t_0) = \big(y(t_0), \ldots, y^{(n-1)}(t_0)\big)^T. For y′′′+2y′′−y′−2y=0y''' + 2y'' - y' - 2y = 0,

X′=(01000121−2)X,X' = \begin{pmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 2 & 1 & -2 \end{pmatrix}X,

and the data y(0)=2y(0) = 2, y′(0)=1y'(0) = 1, y′′(0)=−1y''(0) = -1 become X(0)=(2,1,−1)TX(0) = (2, 1, -1)^T, with the solution y=et+2e−t−e−2ty = e^t + 2e^{-t} - e^{-2t}.

Lessons Systems and conversion

Eigenvalue solutions

For a constant matrix AA, if Av=λvAv = \lambda v with v≠0v \ne 0, then X=eλtvX = e^{\lambda t}v solves X′=AXX' = AX. The eigenvalues are the roots of det⁡(A−λI)=0\det(A - \lambda I) = 0.

  • Real eigenbasis. If AA has nn independent eigenvectors v1,…,vnv_1, \ldots, v_n, which is always the case for nn distinct eigenvalues, then X=c1eλ1tv1+⋯+cneλntvnX = c_1e^{\lambda_1t}v_1 + \cdots + c_ne^{\lambda_nt}v_n, and the constants solve Vc=X(0)Vc = X(0) for the matrix VV with columns v1,…,vnv_1, \ldots, v_n.
  • Complex pair. For a real AA with eigenvalue λ=α+iβ\lambda = \alpha + i\beta, β≠0\beta \ne 0, and eigenvector v=p+iqv = p + iq with pp and qq real, the real and imaginary parts of eλtve^{\lambda t}v are two independent real solutions,
X1=eαt(pcos⁡βt−qsin⁡βt),X2=eαt(psin⁡βt+qcos⁡βt).\begin{aligned} X_1 &= e^{\alpha t}(p\cos\beta t - q\sin\beta t), \\[0.4em] X_2 &= e^{\alpha t}(p\sin\beta t + q\cos\beta t). \end{aligned}

The matrix with rows (−2,1)(-2, 1) and (0,−1)(0, -1) has eigenvalues −2-2 and −1-1 with eigenvectors v1=(1,0)Tv_1 = (1, 0)^T and v2=(1,1)Tv_2 = (1, 1)^T, and X(0)=(3,1)TX(0) = (3, 1)^T gives X=2e−2tv1+e−tv2X = 2e^{-2t}v_1 + e^{-t}v_2. The matrix with rows (−1,−2)(-1, -2) and (2,−1)(2, -1) has eigenvalues −1±2i-1 \pm 2i with v=(1,−i)Tv = (1, -i)^T, and X(0)=(1,0)TX(0) = (1, 0)^T gives X=e−t(cos⁡2t,sin⁡2t)TX = e^{-t}(\cos 2t, \sin 2t)^T, which spirals into the origin counterclockwise.

Lessons Real eigenvalues, Complex eigenvalues

Planar phase portraits

For X′=AXX' = AX with a real 2×22 \times 2 matrix AA, trace τ=tr⁡A\tau = \operatorname{tr} A and determinant Δ=det⁡A\Delta = \det A, the eigenvalues solve λ2−τλ+Δ=0\lambda^2 - \tau\lambda + \Delta = 0.

ConditionEigenvaluesThe origin is
Δ<0\Delta < 0real, opposite signsa saddle, unstable
Δ>0\Delta > 0, τ2>4Δ\tau^2 > 4\Deltareal, distinct, one signa node, stable if τ<0\tau < 0 and unstable if τ>0\tau > 0
Δ>0\Delta > 0, τ2=4Δ\tau^2 = 4\Deltaone repeated real eigenvaluea star node if A=λIA = \lambda I and a degenerate node otherwise, stable if τ<0\tau < 0 and unstable if τ>0\tau > 0
Δ>0\Delta > 0, 0<τ2<4Δ0 < \tau^2 < 4\Deltaα±iβ\alpha \pm i\beta with α≠0\alpha \ne 0a spiral, stable if τ<0\tau < 0 and unstable if τ>0\tau > 0
Δ>0\Delta > 0, τ=0\tau = 0±iβ\pm i\beta with β≠0\beta \ne 0a center, stable but not asymptotically stable
Δ=0\Delta = 000 is an eigenvaluenot isolated, with a whole line of equilibria or, for A=0A = 0, the whole plane

For spirals and centers the motion is counterclockwise when the lower left entry of AA is positive and clockwise when it is negative. The orbits of a center are ellipses in general, and circles only in special coordinates. A saddle has one attracting and one repelling eigenvector line, and almost every orbit eventually follows the repelling one.

Lessons Phase portraits

Repeated eigenvalues and Jordan chains

The algebraic multiplicity of an eigenvalue λ\lambda is its multiplicity as a root of det⁡(A−λI)\det(A - \lambda I), and its geometric multiplicity is dim⁡ker⁡(A−λI)\dim\ker(A - \lambda I), which is never larger. The matrix is diagonalizable exactly when the two agree for every eigenvalue, and defective otherwise. For a defective 2×22 \times 2 matrix with eigenvalue λ\lambda and eigenvector vv, choose a generalized eigenvector ww with (A−λI) w=v(A - \lambda I)\,w = v, which makes vv, ww a Jordan chain. Then

eλtvandeλt(tv+w)e^{\lambda t}v \quad\text{and}\quad e^{\lambda t}(tv + w)

form a fundamental set. Longer chains, with (A−λI) wk+1=wk(A - \lambda I)\,w_{k+1} = w_k, bring in the factors t22!\frac{t^2}{2!}, t33!\frac{t^3}{3!} and so on. For the matrix with rows (2,1)(2, 1) and (0,2)(0, 2), the vectors v=(1,0)Tv = (1, 0)^T and w=(0,1)Tw = (0, 1)^T form a chain, and X(0)=(0,1)TX(0) = (0, 1)^T gives X=e2t(t,1)TX = e^{2t}(t, 1)^T. Writing A=2I+NA = 2I + N with N2=0N^2 = 0 gives etA=e2t(I+tN)e^{tA} = e^{2t}(I + tN). The solution teλtte^{\lambda t} still decays when Re⁡λ<0\operatorname{Re}\lambda < 0, while for Re⁡λ=0\operatorname{Re}\lambda = 0 the factor tt makes it unbounded.

Lessons Repeated eigenvalues

Matrix exponentials and fundamental matrices

For a constant n×nn \times n matrix AA the series

etA=∑k=0∞tkAkk!e^{tA} = \sum_{k=0}^{\infty}\frac{t^kA^k}{k!}

converges for every tt, satisfies ddtetA=AetA\frac{d}{dt}e^{tA} = Ae^{tA} and e0A=Ie^{0A} = I, and gives the solution X(t)=e(t−t0)AX0X(t) = e^{(t - t_0)A}X_0 of X′=AXX' = AX with X(t0)=X0X(t_0) = X_0. The rule eA+B=eAeBe^{A + B} = e^Ae^B holds when AB=BAAB = BA and can fail otherwise. With A1A_1 having rows (0,1)(0, 1) and (0,0)(0, 0) and A2A_2 having rows (0,0)(0, 0) and (1,0)(1, 0), the product eA2eA1e^{A_2}e^{A_1} has rows (1,1)(1, 1) and (1,2)(1, 2), while eA1+A2e^{A_1 + A_2} has cosh⁡1\cosh 1 on the diagonal and sinh⁡1\sinh 1 off it. For the oscillator matrix with rows (0,1)(0, 1) and (−4,0)(-4, 0),

etA=(cos⁡2t12sin⁡2t−2sin⁡2tcos⁡2t).e^{tA} = \begin{pmatrix} \cos 2t & \tfrac{1}{2}\sin 2t \\ -2\sin 2t & \cos 2t \end{pmatrix}.

A fundamental matrix Φ(t)\Phi(t) of X′=A(t) XX' = A(t)\,X has nn independent solutions as its columns, so Φ′=A(t) Φ\Phi' = A(t)\,\Phi and det⁡Φ(t)≠0\det\Phi(t) \ne 0. Every solution is X(t)=Φ(t) Φ(t0)−1X0X(t) = \Phi(t)\,\Phi(t_0)^{-1}X_0, and for a constant matrix etA=Φ(t) Φ(0)−1e^{tA} = \Phi(t)\,\Phi(0)^{-1}. For A(t)A(t) continuous, Liouville’s formula is

det⁡Φ(t)=det⁡Φ(t0)×exp⁡(∫t0ttr⁡A(τ) dτ).\begin{aligned} &\det\Phi(t) = \det\Phi(t_0) \\[0.4em] &\quad \times \exp\left(\int_{t_0}^{t}\operatorname{tr}A(\tau)\,d\tau\right). \end{aligned}

For a time-dependent matrix, X=exp⁡(∫t0tA(τ) dτ)X0X = \exp\big(\int_{t_0}^t A(\tau)\,d\tau\big)X_0 is a solution when A(t) A(s)=A(s) A(t)A(t)\,A(s) = A(s)\,A(t) for all tt and ss, and not in general.

Lessons Matrix exponentials

Forced and time-dependent systems

For X′=A(t) X+b(t)X' = A(t)\,X + b(t) with AA and bb continuous on II and a fundamental matrix Φ\Phi, variation of constants puts X=ΦcX = \Phi c, so that c′=Φ−1bc' = \Phi^{-1}b, and

X(t)=Φ(t)(Φ(t0)−1X0+∫t0tΦ(τ)−1b(τ) dτ).\begin{aligned} &X(t) = \Phi(t)\Big(\Phi(t_0)^{-1}X_0 \\[0.4em] &\quad + \int_{t_0}^{t}\Phi(\tau)^{-1}b(\tau)\,d\tau\Big). \end{aligned}

For a constant matrix this reads

X(t)=e(t−t0)AX0+∫t0te(t−τ)A b(τ) dτ.\begin{aligned} X(t) &= e^{(t - t_0)A}X_0 \\[0.4em] &\quad + \int_{t_0}^{t} e^{(t - \tau)A}\,b(\tau)\,d\tau. \end{aligned}

For A(t)=diag⁡(t,−1)A(t) = \operatorname{diag}(t, -1), b=(1,2)Tb = (1, 2)^T and X(0)=(1,1)TX(0) = (1, 1)^T, a fundamental matrix is Φ=diag⁡(et2/2,e−t)\Phi = \operatorname{diag}(e^{t^2/2}, e^{-t}), which gives x2=2−e−tx_2 = 2 - e^{-t} and

x1=et2/2(1+∫0te−τ2/2 dτ).x_1 = e^{t^2/2}\left(1 + \int_0^t e^{-\tau^2/2}\,d\tau\right).

The integral has no elementary formula, and it equals π/2 erf⁡(t/2)\sqrt{\pi/2}\,\operatorname{erf}(t/\sqrt{2}).

Lessons Forced and time-dependent systems

Coupled oscillators and normal modes

Undamped masses and springs give Mq′′+Kq=0Mq'' + Kq = 0 for the displacement vector qq, with a symmetric positive definite mass matrix MM and a symmetric stiffness matrix KK. The trial q=vcos⁡(ωt−ϕ)q = v\cos(\omega t - \phi) leads to the generalized eigenvalue problem

Kv=ω2Mv.Kv = \omega^2Mv.

When KK is positive definite there are nn positive values ω12,…,ωn2\omega_1^2, \ldots, \omega_n^2, and the mode shapes viv_i can be chosen MM-orthogonal, viTMvj=0v_i^TMv_j = 0 for i≠ji \ne j. The modal coordinates η\eta defined by q=Vηq = V\eta, with the viv_i as the columns of VV, separate the system into ηi′′+ωi2ηi=0\eta_i'' + \omega_i^2\eta_i = 0, so every motion is

q(t)=∑i=1nvi (Aicos⁡ωit+Bisin⁡ωit).\begin{aligned} &q(t) = \sum_{i=1}^{n} v_i\,(A_i\cos\omega_it \\[0.4em] &\qquad + B_i\sin\omega_it). \end{aligned}

A zero eigenvalue, possible when KK is only positive semidefinite, gives a rigid motion ηi=Ai+Bit\eta_i = A_i + B_it instead. With M=IM = I and KK having rows (3,−1)(3, -1) and (−1,3)(-1, 3), the in-phase mode (1,1)T(1, 1)^T has ω2=2\omega^2 = 2 and the opposite mode (1,−1)T(1, -1)^T has ω2=4\omega^2 = 4. Releasing q(0)=(2,0)Tq(0) = (2, 0)^T from rest gives q1=cos⁡2 t+cos⁡2tq_1 = \cos\sqrt{2}\,t + \cos 2t and q2=cos⁡2 t−cos⁡2tq_2 = \cos\sqrt{2}\,t - \cos 2t, and the energy 12(q′)TMq′+12qTKq=6\frac{1}{2}(q')^TMq' + \frac{1}{2}q^TKq = 6 stays constant.

Lessons Coupled oscillators and normal modes

Nonlinear Dynamics and Stability

Most nonlinear systems have no solution formula, so their behavior is read from the geometry of their solutions. Linearization decides stability near most equilibria and Lyapunov functions handle many of the rest. As a parameter changes, equilibria and periodic orbits can appear, disappear or change stability, and in three or more dimensions bounded solutions can be chaotic.

Phase planes and nullclines

For an autonomous planar system x′=f(x,y)x' = f(x, y), y′=g(x,y)y' = g(x, y) with ff and gg continuously differentiable on an open set, the vector field F=(f,g)TF = (f, g)^T gives the direction of motion at each point.

  • The xx-nullcline is the set where f=0f = 0, and the flow crosses it vertically. The yy-nullcline is where g=0g = 0, and the flow crosses it horizontally.
  • Equilibria are the points where an xx-nullcline meets a yy-nullcline.
  • By uniqueness, trajectories never cross each other and never reach an equilibrium in finite time. A nullcline is usually not a trajectory.
  • A conserved quantity HH, with ∇H⋅F=0\nabla H \cdot F = 0, is constant along trajectories, so trajectories lie on its level curves.

The predator and prey model x′=x(2−y)x' = x(2 - y), y′=y(x−1)y' = y(x - 1) on x,y≥0x, y \ge 0 has the xx-nullclines x=0x = 0 and y=2y = 2, the yy-nullclines y=0y = 0 and x=1x = 1, and the equilibria (0,0)(0, 0) and (1,2)(1, 2). For x,y>0x, y > 0 the quantity H=x−ln⁡x+y−2ln⁡yH = x - \ln x + y - 2\ln y is conserved and has its minimum at (1,2)(1, 2), so the orbits around (1,2)(1, 2) are closed curves.

Lessons Phase planes and nullclines

Linearization

Near an equilibrium X∗X_* of X′=F(X)X' = F(X), with FF continuously differentiable, the displacement h=X−X∗h = X - X_* approximately satisfies h′=Jhh' = Jh, where JJ is the Jacobian matrix of FF at X∗X_*, with entries ∂Fi/∂xj\partial F_i/\partial x_j. The equilibrium is hyperbolic when no eigenvalue of JJ has zero real part.

  • If every eigenvalue has negative real part, X∗X_* is locally asymptotically stable.
  • If some eigenvalue has positive real part, X∗X_* is unstable.
  • At a hyperbolic equilibrium, the Hartman-Grobman theorem says that near X∗X_* a continuous change of coordinates carries the orbits of the nonlinear system onto those of the linearization, keeping the direction of time.
  • At a nonhyperbolic equilibrium linearization decides nothing. Both x′=−x3x' = -x^3 and x′=x3x' = x^3 have J=0J = 0 at the origin, yet the first attracts and the second repels.

These results are local, and the theorem does not say how large the neighborhood is. For x′=x(1−x)x' = x(1 - x), y′=−2yy' = -2y, the Jacobian is diag⁡(1−2x,−2)\operatorname{diag}(1 - 2x, -2), so (1,0)(1, 0) with J=diag⁡(−1,−2)J = \operatorname{diag}(-1, -2) is a sink and (0,0)(0, 0) with J=diag⁡(1,−2)J = \operatorname{diag}(1, -2) is a saddle. Starting from x(0)=−1x(0) = -1, though, x=11−2e−tx = \frac{1}{1 - 2e^{-t}} blows up at t=ln⁡2t = \ln 2.

Lessons Linearization and local stability

Lyapunov functions

Let the origin be an equilibrium of X′=F(X)X' = F(X) and let VV be continuously differentiable on a neighborhood UU of the origin and positive definite, meaning V(0)=0V(0) = 0 and V(X)>0V(X) > 0 for X≠0X \ne 0 in UU. Its derivative along solutions is the orbital derivative V˙=∇V⋅F\dot V = \nabla V \cdot F.

On UU without the originConclusion
V˙≤0\dot V \le 0stable
V˙<0\dot V < 0asymptotically stable
V˙<0\dot V < 0, with UU the whole space and V→∞V \to \infty as ∥X∥→∞\lVert X \rVert \to \inftyglobally asymptotically stable

LaSalle’s invariance principle. If KK is a compact positively invariant set, one that no solution starting in it leaves in forward time, and V˙≤0\dot V \le 0 on KK, then every solution starting in KK approaches the largest invariant set contained in {X∈K:V˙(X)=0}\{X \in K : \dot V(X) = 0\}. Here VV need not be positive definite.

For x′=yx' = y, y′=−x−x3−yy' = -x - x^3 - y take V=12y2+12x2+14x4V = \frac{1}{2}y^2 + \frac{1}{2}x^2 + \frac{1}{4}x^4. Then V˙=−y2≤0\dot V = -y^2 \le 0, which vanishes on the whole line y=0y = 0. A solution staying on that line needs y′=−x−x3=0y' = -x - x^3 = 0, so x=0x = 0, and the only invariant set there is the origin. The sublevel sets of VV are compact and are never left, so the origin is globally asymptotically stable. Without damping, x′=yx' = y and y′=−xy' = -x keep VV constant, and the origin is stable but not asymptotically stable. A candidate VV that fails these tests proves nothing either way.

Lessons Lyapunov functions

One-dimensional bifurcations

For x′=f(x;μ)x' = f(x; \mu) with ff smooth, an equilibrium x∗x_* is stable when fx(x∗;μ)<0f_x(x_*; \mu) < 0 and unstable when fx(x∗;μ)>0f_x(x_*; \mu) > 0. The number of equilibria near x∗x_* can change only at a parameter value where f=0f = 0 and fx=0f_x = 0 hold together. A bifurcation diagram plots the equilibria against μ\mu and marks which are stable.

TypeNormal formEquilibriaStability
Saddle-nodex′=μ−x2x' = \mu - x^2±μ\pm\sqrt{\mu} for μ>0\mu > 0, none for μ<0\mu < 0μ\sqrt{\mu} stable, −μ-\sqrt{\mu} unstable
Transcriticalx′=μx−x2x' = \mu x - x^200 and μ\muthey exchange stability at μ=0\mu = 0, and the larger one is stable
Supercritical pitchforkx′=μx−x3x' = \mu x - x^300, and ±μ\pm\sqrt{\mu} for μ>0\mu > 000 stable for μ<0\mu < 0 and unstable for μ>0\mu > 0, ±μ\pm\sqrt{\mu} stable
Subcritical pitchforkx′=μx+x3x' = \mu x + x^300, and ±−μ\pm\sqrt{-\mu} for μ<0\mu < 000 stable for μ<0\mu < 0 and unstable for μ>0\mu > 0, ±−μ\pm\sqrt{-\mu} unstable

A pitchfork is typical only for equations with the reflection symmetry f(−x;μ)=−f(x;μ)f(-x; \mu) = -f(x; \mu), and without it a small change usually breaks it into other bifurcations. For μ=1\mu = 1 the supercritical form x′=x−x3x' = x - x^3 with x(0)=12x(0) = \frac{1}{2} has the solution x=(1+3e−2t)−1/2x = (1 + 3e^{-2t})^{-1/2}, which tends to the stable branch 11.

Lessons One-dimensional bifurcations

Limit cycles

A periodic orbit is the closed path of a nonconstant periodic solution, and a limit cycle is a periodic orbit with no other periodic orbit in some neighborhood of it. The omega-limit set of a trajectory is the set of points it approaches along some sequence of times tending to ∞\infty.

  • Poincaré-Bendixson theorem. For a planar system with FF continuously differentiable on an open set DD, if a forward trajectory stays in a compact set K⊆DK \subseteq D and its omega-limit set contains no equilibrium, that omega-limit set is a periodic orbit. In particular, a compact region with no equilibria that no forward trajectory leaves, a trapping region, contains a periodic orbit.
  • Bendixson-Dulac criterion. If DD is a simply connected open set in the plane and some continuously differentiable function BB makes ∇⋅(BF)=(Bf)x+(Bg)y\nabla \cdot (BF) = (Bf)_x + (Bg)_y positive everywhere on DD or negative everywhere on DD, then no periodic orbit lies entirely in DD. With B=1B = 1 this is Bendixson’s criterion.

Both results are about the plane and do not carry over to three dimensions. The system x′=x(1−x2−y2)−yx' = x(1 - x^2 - y^2) - y, y′=y(1−x2−y2)+xy' = y(1 - x^2 - y^2) + x becomes r′=r(1−r2)r' = r(1 - r^2), θ′=1\theta' = 1 in polar coordinates. The unit circle, traced by (cos⁡t,sin⁡t)(\cos t, \sin t), is an attracting limit cycle, and the annulus 12≤r≤32\frac{1}{2} \le r \le \frac{3}{2} is a trapping region without equilibria. The damped oscillator x′=yx' = y, y′=−x−yy' = -x - y has divergence −1-1 everywhere, so it has no periodic orbit.

Lessons Limit cycles

Hopf bifurcation

In the normal form

x′=μx−y−x(x2+y2),y′=x+μy−y(x2+y2),\begin{aligned} x' &= \mu x - y - x(x^2 + y^2), \\[0.4em] y' &= x + \mu y - y(x^2 + y^2), \end{aligned}

the origin has eigenvalues μ±i\mu \pm i, and polar coordinates give r′=μr−r3r' = \mu r - r^3 and θ′=1\theta' = 1. For μ≤0\mu \le 0 the origin is asymptotically stable, at μ=0\mu = 0 only slowly, with r=r0/1+2r02tr = r_0/\sqrt{1 + 2r_0^2t}. For μ>0\mu > 0 the origin is unstable and the circle r=μr = \sqrt{\mu} is an attracting limit cycle of period 2π2\pi. This is a supercritical Hopf bifurcation. Reversing the signs of the cubic terms gives r′=μr+r3r' = \mu r + r^3, the subcritical case, where an unstable cycle r=−μr = \sqrt{-\mu} surrounds the stable origin for μ<0\mu < 0.

Hopf bifurcation theorem. Take a sufficiently smooth planar family X′=F(X;μ)X' = F(X; \mu) with an equilibrium X∗(μ)X_*(\mu) whose Jacobian has eigenvalues α(μ)±iβ(μ)\alpha(\mu) \pm i\beta(\mu), and suppose

  • the eigenvalues reach the imaginary axis, α(μ0)=0\alpha(\mu_0) = 0 with β(μ0)≠0\beta(\mu_0) \ne 0,
  • the transversality condition α′(μ0)≠0\alpha'(\mu_0) \ne 0 holds,
  • the nondegeneracy condition holds, the first Lyapunov coefficient ℓ1\ell_1 being nonzero.

Then near X∗(μ0)X_*(\mu_0) and μ0\mu_0 a unique family of periodic orbits branches off on one side of μ0\mu_0, with amplitude proportional to ∣μ−μ0∣\sqrt{\lvert \mu - \mu_0 \rvert} and period near 2π/β(μ0)2\pi/\beta(\mu_0). If ℓ1<0\ell_1 < 0 the orbits are stable and exist where the equilibrium is unstable, the supercritical case. If ℓ1>0\ell_1 > 0 they are unstable and exist where the equilibrium is stable, the subcritical case. The theorem says nothing about large oscillations away from X∗X_*.

Lessons Hopf bifurcation

Chaos and sensitive dependence

A common working definition of chaos is bounded, aperiodic long-term behavior of a deterministic system with sensitive dependence on initial conditions. An autonomous system needs at least three dimensions for it, because the Poincaré-Bendixson theorem restricts bounded planar trajectories. The Lorenz system

x′=10(y−x),y′=x(28−z)−y,z′=xy−83z\begin{aligned} x' &= 10(y - x), \\[0.4em] y' &= x(28 - z) - y, \\[0.4em] z' &= xy - \tfrac{8}{3}z \end{aligned}

has the equilibria (0,0,0)(0, 0, 0) and (±62,±62,27)(\pm 6\sqrt{2}, \pm 6\sqrt{2}, 27), with matching signs. The origin is a saddle with eigenvalues −11±12012\frac{-11 \pm \sqrt{1201}}{2} and −83-\frac{8}{3}. The divergence of the field is −413-\frac{41}{3} everywhere, so phase-space volumes shrink like e−41t/3e^{-41t/3} and bounded solutions approach an attractor of zero volume. Nearby solutions on it separate roughly like eλ1te^{\lambda_1t} with largest Lyapunov exponent λ1≈0.9\lambda_1 \approx 0.9, a numerical estimate.

A periodically forced planar equation such as the Duffing equation q′′+δq′+αq+βq3=γcos⁡ωtq'' + \delta q' + \alpha q + \beta q^3 = \gamma\cos\omega t is autonomous in the three variables qq, q′q' and θ=ωt\theta = \omega t, and it can be chaotic. A computed trajectory is evidence, not proof, because rounding and truncation errors grow at the same exponential rate as the true separation.

Lessons Chaos and sensitive dependence

Numerical Methods and Reliable Computation

A numerical method follows the solution of y′=f(t,y)y' = f(t, y), y(t0)=y0y(t_0) = y_0 in steps of size hh, with tn=t0+nht_n = t_0 + nh and yn≈y(tn)y_n \approx y(t_n). The order of accuracy says how fast the error falls as hh shrinks, and stability says whether errors are damped or amplified. Neither alone guarantees a trustworthy answer, so a computed solution is always checked.

Euler’s method and error

Euler’s method follows the tangent line for one step,

yn+1=yn+h f(tn,yn).y_{n+1} = y_n + h\,f(t_n, y_n).
  • The local truncation error, made in one step that starts from the exact value, is of order h2h^2 when y′′y'' is bounded. Some books divide it by hh and call it of order hh.
  • The global error yn−y(tn)y_n - y(t_n) on a fixed interval is of order hh. If ff is Lipschitz in yy with constant L>0L > 0 and ∣y′′∣≤K\lvert y'' \rvert \le K, then
∣yn−y(tn)∣≤Kh2L×(eL(tn−t0)−1).\begin{aligned} &\lvert y_n - y(t_n) \rvert \le \frac{Kh}{2L} \\[0.4em] &\quad \times \left(e^{L(t_n - t_0)} - 1\right). \end{aligned}

Halving hh roughly halves the error once hh is small. For y′=−2yy' = -2y with y(0)=1y(0) = 1, the step h=14h = \frac{1}{4} gives y2=14y_2 = \frac{1}{4} at t=12t = \frac{1}{2} against e−1≈0.368e^{-1} \approx 0.368, and h=18h = \frac{1}{8} gives (34)4=81256≈0.316(\frac{3}{4})^4 = \frac{81}{256} \approx 0.316. Each step multiplies by 1−2h1 - 2h, so the computed values decay only for 0<h<10 < h < 1 and stay positive only for h<12h < \frac{1}{2}.

Lessons Euler’s method

Runge-Kutta methods

Each stage kik_i is a slope, and the step combines them.

MethodStagesStepGlobal error
Eulerk1=f(tn,yn)k_1 = f(t_n, y_n)yn+hk1y_n + hk_1order hh
Heun, or improved Eulerk1k_1 and k2=f(tn+h,yn+hk1)k_2 = f(t_n + h, y_n + hk_1)yn+h2(k1+k2)y_n + \frac{h}{2}(k_1 + k_2)order h2h^2
Midpointk1k_1 and k2=f(tn+h2,yn+h2k1)k_2 = f(t_n + \frac{h}{2}, y_n + \frac{h}{2}k_1)yn+hk2y_n + hk_2order h2h^2
Classical RK4the four stages belowyn+h6(k1+2k2+2k3+k4)y_n + \frac{h}{6}(k_1 + 2k_2 + 2k_3 + k_4)order h4h^4
k1=f(tn,yn),k2=f(tn+h2, yn+h2k1),k3=f(tn+h2, yn+h2k2),k4=f(tn+h, yn+hk3).\begin{aligned} k_1 &= f(t_n, y_n), \\[0.4em] k_2 &= f\big(t_n + \tfrac{h}{2},\, y_n + \tfrac{h}{2}k_1\big), \\[0.4em] k_3 &= f\big(t_n + \tfrac{h}{2},\, y_n + \tfrac{h}{2}k_2\big), \\[0.4em] k_4 &= f(t_n + h,\, y_n + hk_3). \end{aligned}

On smooth problems, halving hh divides the global error of a method of order pp by about 2p2^p, so by 22 for Euler, 44 for Heun and midpoint and 1616 for RK4. One step of h=0.1h = 0.1 for y′=yy' = y, y(0)=1y(0) = 1 gives

Methody1y_1Error against e0.1=1.1051709e^{0.1} = 1.1051709
Euler1.11.15.2×10−35.2 \times 10^{-3}
Heun or midpoint1.1051.1051.7×10−41.7 \times 10^{-4}
RK41.10517081.10517088.5×10−88.5 \times 10^{-8}

Heun and midpoint agree on this equation but not in general. For y′=t2y' = t^2 from y(0)=0y(0) = 0, one step gives h32\frac{h^3}{2} and h34\frac{h^3}{4} against the exact h33\frac{h^3}{3}.

Adaptive steps. For RK4, comparing one step of size hh with two steps of size h/2h/2 gives the error estimate yh/2−yh15\frac{y_{h/2} - y_h}{15} for the more accurate value yh/2y_{h/2}. Embedded pairs get two approximations of different orders from shared stages and use their difference as the local error estimate. A step is accepted when that estimate is at most atol+rtol ∣y∣\text{atol} + \text{rtol}\,\lvert y \rvert, the absolute tolerance plus the relative tolerance times the size of the solution, and the next step is scaled by about (tol/err)1/(p+1)(\text{tol}/\text{err})^{1/(p+1)} with a safety factor, where pp is the lower order. Tolerances control the estimated local error, not the global error.

Lessons Heun and midpoint methods, RK4 and adaptive steps

Stiffness and implicit methods

Stability is studied on the test equation y′=λyy' = \lambda y with Re⁡λ<0\operatorname{Re}\lambda < 0, whose solutions decay. One step multiplies yny_n by an amplification factor R(z)R(z) with z=hλz = h\lambda, and the region of absolute stability is where ∣R(z)∣<1\lvert R(z) \rvert < 1.

MethodR(z)R(z)Decay for real λ<0\lambda < 0 when
Explicit Euler1+z1 + zh<2∣λ∣h < \dfrac{2}{\lvert \lambda \rvert}
Classical RK41+z+z22+z36+z4241 + z + \dfrac{z^2}{2} + \dfrac{z^3}{6} + \dfrac{z^4}{24}h<2.785∣λ∣h < \dfrac{2.785}{\lvert \lambda \rvert}, approximately
Implicit Euler11−z\dfrac{1}{1 - z}every h>0h > 0

Implicit Euler uses the slope at the new point, yn+1=yn+h f(tn+1,yn+1)y_{n+1} = y_n + h\,f(t_{n+1}, y_{n+1}), so each step solves an equation for yn+1y_{n+1}, by Newton’s method when ff is nonlinear. A method is A-stable when its stability region contains the whole left half-plane Re⁡z<0\operatorname{Re} z < 0. Implicit Euler is A-stable, and no explicit Runge-Kutta method is.

An equation is stiff when fast decaying modes force an explicit method to take steps far smaller than accuracy for the slow behavior would need. For y′=−1000(y−cos⁡t)−sin⁡ty' = -1000(y - \cos t) - \sin t with y(0)=1y(0) = 1, the solution is cos⁡t\cos t, and nearby solutions approach it like e−1000te^{-1000t}. With h=0.01h = 0.01 a deviation is multiplied by −9-9 in an explicit Euler step and by 111\frac{1}{11} in an implicit Euler step, against the exact factor e−10≈4.5×10−5e^{-10} \approx 4.5 \times 10^{-5}. Stability is not accuracy, and implicit Euler is still only of order hh.

Lessons Stiffness and implicit methods

Boundary value problems

For y′′=f(x,y,y′)y'' = f(x, y, y') on [a,b][a, b] with y(a)=αy(a) = \alpha and y(b)=βy(b) = \beta, in the variable xx, two methods are standard.

  • Shooting. Solve the initial value problem with y(a)=αy(a) = \alpha and y′(a)=σy'(a) = \sigma, and adjust σ\sigma until the residual R(σ)=y(b;σ)−βR(\sigma) = y(b; \sigma) - \beta is zero, by bisection, the secant method or Newton’s method.
  • Finite differences. On the grid xi=a+ihx_i = a + ih replace y′′(xi)y''(x_i) by
Yi+1−2Yi+Yi−1h2,\frac{Y_{i+1} - 2Y_i + Y_{i-1}}{h^2},

which has error of order h2h^2, and solve the resulting system, which is tridiagonal for a linear equation.

For y′′=−2y'' = -2 with y(0)=y(1)=0y(0) = y(1) = 0 and h=14h = \frac{1}{4}, the system 2Yi−Yi−1−Yi+1=182Y_i - Y_{i-1} - Y_{i+1} = \frac{1}{8} gives (Y1,Y2,Y3)=(316,14,316)(Y_1, Y_2, Y_3) = (\frac{3}{16}, \frac{1}{4}, \frac{3}{16}), exactly the values of y=x−x2y = x - x^2, because the difference quotient is exact for quadratics. When the homogeneous problem has a nonzero solution, as Asin⁡πxA\sin\pi x does for y′′+π2y=0y'' + \pi^2y = 0 with y(0)=y(1)=0y(0) = y(1) = 0, the boundary problem has no solution or infinitely many. Numerically this shows up as a nearly singular system or a shooting residual that does not depend on σ\sigma.

Lessons Numerical boundary value problems

Checking a computed solution

  • Residual. Substitute the computed solution into the equation and check that the difference between the two sides is small.
  • Data. Check the initial and boundary values separately.
  • Refinement. Halve the step or tighten the tolerances and check that the answer settles. With an error EhE_h against a trusted value, the observed order is p≈log⁡2(Eh/Eh/2)p \approx \log_2(E_h/E_{h/2}).
  • Units and invariants. Every term must have the same units, and conserved quantities such as energy or total mass should stay constant to the expected accuracy.
  • Events and singularities. An event is a time at which a stated condition first holds, located during the integration. A singularity is where the solution stops existing, and no solver can integrate through it.

For y′=y2y' = y^2 with y(0)=1y(0) = 1, the solution 11−t\frac{1}{1 - t} reaches the event y=4y = 4 at t=34t = \frac{3}{4} and blows up at t=1t = 1. Euler with h=14h = \frac{1}{4} gives Y1=54Y_1 = \frac{5}{4} and Y2=10564≈1.64Y_2 = \frac{105}{64} \approx 1.64 against y(12)=2y(\frac{1}{2}) = 2. Near t=1t = 1 refinement stops settling, which signals the singularity.

Lessons Reliable computation

Boundary Problems and Eigenvalue Methods

Conditions at both ends of an interval behave differently from initial conditions. A linear boundary problem has exactly one solution, none or infinitely many, depending on whether the homogeneous problem has a nonzero solution. The parameter values that produce such solutions are eigenvalues, and their eigenfunctions are the building blocks of Fourier series and separation of variables. In this unit the independent variable is xx.

Solvability

Take a linear second-order equation L[y]=fL[y] = f on [a,b][a, b] with continuous coefficients, a nonzero leading coefficient and two homogeneous linear boundary conditions.

  • If the homogeneous problem, with f=0f = 0, has only the zero solution, there is exactly one solution for every continuous ff.
  • If it has a nonzero solution φ\varphi, the problem is resonant and has no solution or infinitely many. For self-adjoint problems such as y′′+y=fy'' + y = f with zero end values, a solution exists exactly when the compatibility condition ∫abf φ dx=0\int_a^b f\,\varphi\,dx = 0 holds, and then the solutions are yp+Cφy_p + C\varphi.

For y′′+y=fy'' + y = f on [0,π][0, \pi] with y(0)=y(π)=0y(0) = y(\pi) = 0, the homogeneous solution is sin⁡x\sin x and the condition is ∫0πf(x)sin⁡x dx=0\int_0^\pi f(x)\sin x\,dx = 0. The forcing sin⁡x\sin x fails it, since the particular solution −12xcos⁡x-\frac{1}{2}x\cos x has the value π2\frac{\pi}{2} at π\pi and no multiple of sin⁡x\sin x can correct that. The forcing cos⁡x\cos x passes, with the solutions 12xsin⁡x+Csin⁡x\frac{1}{2}x\sin x + C\sin x. On [0,π2][0, \frac{\pi}{2}] with zero end values the homogeneous problem has only the zero solution, and f=1f = 1 gives the single solution 1−sin⁡x−cos⁡x1 - \sin x - \cos x.

Lessons Solvability

Eigenvalues and eigenfunctions

An eigenvalue problem asks for the numbers λ\lambda for which −y′′=λy-y'' = \lambda y with homogeneous boundary conditions has a nonzero solution, an eigenfunction. Every nonzero multiple of an eigenfunction is again one, and the cases λ<0\lambda < 0, λ=0\lambda = 0 and λ>0\lambda > 0 are checked separately. On [0,ℓ][0, \ell],

Boundary conditionsEigenvaluesEigenfunctions
Dirichlet, y(0)=y(ℓ)=0y(0) = y(\ell) = 0(nπℓ)2\left(\dfrac{n\pi}{\ell}\right)^2, n=1,2,…n = 1, 2, \ldotssin⁡nπxℓ\sin\dfrac{n\pi x}{\ell}
Neumann, y′(0)=y′(ℓ)=0y'(0) = y'(\ell) = 0(nπℓ)2\left(\dfrac{n\pi}{\ell}\right)^2, n=0,1,2,…n = 0, 1, 2, \ldotscos⁡nπxℓ\cos\dfrac{n\pi x}{\ell}, the constant for n=0n = 0
Mixed, y(0)=0y(0) = 0 and y′(ℓ)=0y'(\ell) = 0((2n−1)π2ℓ)2\left(\dfrac{(2n - 1)\pi}{2\ell}\right)^2, n=1,2,…n = 1, 2, \ldotssin⁡(2n−1)πx2ℓ\sin\dfrac{(2n - 1)\pi x}{2\ell}
Periodic, y(0)=y(ℓ)y(0) = y(\ell) and y′(0)=y′(ℓ)y'(0) = y'(\ell)(2nπℓ)2\left(\dfrac{2n\pi}{\ell}\right)^2, n=0,1,2,…n = 0, 1, 2, \ldotsthe constant, and both cos⁡2nπxℓ\cos\dfrac{2n\pi x}{\ell} and sin⁡2nπxℓ\sin\dfrac{2n\pi x}{\ell} for n≥1n \ge 1

The Neumann and periodic problems have the zero eigenvalue with a constant eigenfunction, the zero mode, and each periodic eigenvalue with n≥1n \ge 1 has two independent eigenfunctions. The Robin conditions y′(0)=h0 y(0)y'(0) = h_0\,y(0) and y′(ℓ)=−hℓ y(ℓ)y'(\ell) = -h_\ell\,y(\ell), with the signs set by the outward direction at each end, lead to the identity

λ∫0ℓy2 dx=∫0ℓ(y′)2 dx+h0 y(0)2+hℓ y(ℓ)2.\begin{aligned} &\lambda\int_0^\ell y^2\,dx = \int_0^\ell (y')^2\,dx \\[0.4em] &\quad + h_0\,y(0)^2 + h_\ell\,y(\ell)^2. \end{aligned}

So for h0,hℓ≥0h_0, h_\ell \ge 0, not both zero, every eigenvalue is positive. They are λ=k2\lambda = k^2 with k>0k > 0 solving (h0+hℓ) kcos⁡kℓ=(k2−h0hℓ)sin⁡kℓ(h_0 + h_\ell)\,k\cos k\ell = (k^2 - h_0h_\ell)\sin k\ell, with eigenfunctions kcos⁡kx+h0sin⁡kxk\cos kx + h_0\sin kx. A normalized eigenfunction has ∫0ℓφ2 dx=1\int_0^\ell \varphi^2\,dx = 1, as 2/ℓ sin⁡nπxℓ\sqrt{2/\ell}\,\sin\frac{n\pi x}{\ell} does in the Dirichlet case.

Lessons Eigenvalues and eigenfunctions

Regular Sturm-Liouville problems

A regular Sturm-Liouville problem on a finite interval [a,b][a, b] is

−(p y′)′+q y=λ w y,-(p\,y')' + q\,y = \lambda\,w\,y,

where pp is continuously differentiable with p>0p > 0 on [a,b][a, b], qq and ww are continuous with w>0w > 0 on [a,b][a, b], and the boundary conditions are separated,

α0 y(a)+β0 p(a) y′(a)=0,α1 y(b)+β1 p(b) y′(b)=0,\begin{gathered} \alpha_0\,y(a) + \beta_0\,p(a)\,y'(a) = 0, \\[0.4em] \alpha_1\,y(b) + \beta_1\,p(b)\,y'(b) = 0, \end{gathered}

with real coefficients and neither pair (αi,βi)(\alpha_i, \beta_i) equal to (0,0)(0, 0). Under these hypotheses

  • the eigenvalues are real and simple and form an increasing sequence λ1<λ2<⋯\lambda_1 < \lambda_2 < \cdots that tends to ∞\infty, though the first ones may be negative,
  • eigenfunctions with different eigenvalues are orthogonal with the weight ww, so ⟨u,v⟩w=∫abw u v dx=0\langle u, v \rangle_w = \int_a^b w\,u\,v\,dx = 0,
  • the eigenfunctions are complete, so every square-integrable ff has an expansion f=∑cnφnf = \sum c_n\varphi_n in the weighted mean-square sense, with cn=⟨f,φn⟩w/⟨φn,φn⟩wc_n = \langle f, \varphi_n \rangle_w/\langle \varphi_n, \varphi_n \rangle_w.

The problem −y′′−2y=λy-y'' - 2y = \lambda y with Dirichlet conditions on [0,π][0, \pi] has λ1=−1\lambda_1 = -1 with y=sin⁡xy = \sin x, which shows that positive pp and ww do not make every eigenvalue positive. With p=w=exp = w = e^x, q=0q = 0 and y(0)=y(π)=0y(0) = y(\pi) = 0, the eigenvalues are n2+14n^2 + \frac{1}{4} with eigenfunctions e−x/2sin⁡nxe^{-x/2}\sin nx, orthogonal with the weight exe^x. Legendre’s equation, where p=1−x2p = 1 - x^2 vanishes at ±1\pm 1, is singular, and periodic conditions are not separated and allow double eigenvalues, so the regular theorem does not apply to either as stated.

Lessons Regular Sturm-Liouville problems

Green’s functions and the Fredholm alternative

When the homogeneous problem has only the zero solution, the solution of L[y]=fL[y] = f with homogeneous boundary conditions is an integral against a Green’s function, y(x)=∫abG(x,s) f(s) dsy(x) = \int_a^b G(x, s)\,f(s)\,ds. For −y′′=f-y'' = f on [0,1][0, 1] with y(0)=y(1)=0y(0) = y(1) = 0,

G(x,s)={x(1−s),x≤s,s(1−x),x≥s.G(x, s) = \begin{cases} x(1 - s), & x \le s, \\ s(1 - x), & x \ge s. \end{cases}

As a function of xx, the kernel is continuous, solves the homogeneous equation for x≠sx \ne s, satisfies both boundary conditions and has a derivative jump Gx(s+,s)−Gx(s−,s)=−1G_x(s^+, s) - G_x(s^-, s) = -1. For f=2f = 2 it gives y=x(1−x)y = x(1 - x). For −(p y′)′+q y=f-(p\,y')' + q\,y = f with separated conditions, let u1u_1 solve the homogeneous equation with the left condition and u2u_2 with the right one. Then p Wp\,W is constant for W=u1u2′−u1′u2W = u_1u_2' - u_1'u_2, and

G(x,s)=−u1(x<) u2(x>)p W,G(x, s) = -\frac{u_1(x_<)\,u_2(x_>)}{p\,W},

with x<=min⁡(x,s)x_< = \min(x, s) and x>=max⁡(x,s)x_> = \max(x, s). The derivative jump at x=sx = s is then −1/p(s)-1/p(s).

Fredholm alternative. For −(p y′)′+q y−λw y=f-(p\,y')' + q\,y - \lambda w\,y = f with separated conditions on a regular interval, exactly one of two cases holds. Either λ\lambda is not an eigenvalue and there is exactly one solution for each continuous ff, or λ\lambda is an eigenvalue with eigenfunction φ\varphi, and then a solution exists exactly when ∫abf φ dx=0\int_a^b f\,\varphi\,dx = 0, and adding multiples of φ\varphi gives all solutions. If the forcing is written as f=w gf = w\,g, the condition reads ∫abw g φ dx=0\int_a^b w\,g\,\varphi\,dx = 0. For −y′′−π2y=f-y'' - \pi^2y = f on [0,1][0, 1] with zero end values, φ=sin⁡πx\varphi = \sin\pi x. The forcing sin⁡πx\sin\pi x has no solution, and cos⁡πx\cos\pi x has the solutions −xsin⁡πx2π+Csin⁡πx-\frac{x\sin\pi x}{2\pi} + C\sin\pi x.

Lessons Green’s functions and the Fredholm alternative

Beams

An Euler-Bernoulli beam with constant bending stiffness EIEI and mass per unit length ρA\rho A has transverse displacement u(x,t)u(x, t) with

ρA utt+EI uxxxx=0.\rho A\,u_{tt} + EI\,u_{xxxx} = 0.

Vibrations u=X(x)cos⁡ωtu = X(x)\cos\omega t need X′′′′=β4XX'''' = \beta^4X with β4=ρA ω2/EI\beta^4 = \rho A\,\omega^2/EI, where primes mean d/dxd/dx, so XX is a combination of cos⁡βx\cos\beta x, sin⁡βx\sin\beta x, cosh⁡βx\cosh\beta x and sinh⁡βx\sinh\beta x.

EndConditionsMeaning
ClampedX=X′=0X = X' = 0no displacement and no rotation
PinnedX=X′′=0X = X'' = 0no displacement and no bending moment
FreeX′′=X′′′=0X'' = X''' = 0no bending moment and no shear force

For a beam of length ℓ\ell, the frequencies are ω=β2EI/(ρA)\omega = \beta^2\sqrt{EI/(\rho A)}, with β\beta from

SupportsFrequency equationSmallest positive βℓ\beta\ell
pinned at both endssin⁡βℓ=0\sin\beta\ell = 0π\pi
clamped at both endscos⁡βℓ cosh⁡βℓ=1\cos\beta\ell\,\cosh\beta\ell = 14.73004.7300
clamped and freecos⁡βℓ cosh⁡βℓ=−1\cos\beta\ell\,\cosh\beta\ell = -11.87511.8751

A pinned beam has the modes sin⁡nπxℓ\sin\frac{n\pi x}{\ell} with ωn=(nπℓ)2EI/(ρA)\omega_n = \left(\frac{n\pi}{\ell}\right)^2\sqrt{EI/(\rho A)}, and a free beam also has the zero-frequency rigid modes X=1X = 1 and X=xX = x. With any combination of these end conditions, integration by parts gives EI∫0ℓ(X′′)2 dx=ρA ω2∫0ℓX2 dxEI\int_0^\ell (X'')^2\,dx = \rho A\,\omega^2\int_0^\ell X^2\,dx, so ω2≥0\omega^2 \ge 0.

Lessons Beams and higher-order problems

Fourier Methods and Partial Differential Equations

A Fourier series writes a function as a sum of sines and cosines, the eigenfunctions of the boundary problems above. With separation of variables these series solve the heat, wave and Laplace equations on bounded regions, characteristics solve first-order transport equations, and the Fourier transform does the same work on the whole line. In this unit utu_t, uxu_x and uxxu_{xx} are partial derivatives.

Fourier series

For a function ff of period 2L2L, integrable on [−L,L][-L, L], the Fourier series is

f(x)∼a02+∑n=1∞ancos⁡nπxL+∑n=1∞bnsin⁡nπxL,\begin{aligned} &f(x) \sim \frac{a_0}{2} \\[0.4em] &\quad + \sum_{n=1}^{\infty} a_n\cos\frac{n\pi x}{L} \\[0.4em] &\quad + \sum_{n=1}^{\infty} b_n\sin\frac{n\pi x}{L}, \end{aligned}

with the coefficients, for n≥0n \ge 0 and n≥1n \ge 1 respectively,

an=1L∫−LLf(x)cos⁡nπxL dx,bn=1L∫−LLf(x)sin⁡nπxL dx.\begin{aligned} a_n &= \frac{1}{L}\int_{-L}^{L} f(x)\cos\frac{n\pi x}{L}\,dx, \\[0.4em] b_n &= \frac{1}{L}\int_{-L}^{L} f(x)\sin\frac{n\pi x}{L}\,dx. \end{aligned}

With the inner product ⟨f,g⟩=∫−LLf g dx\langle f, g \rangle = \int_{-L}^{L} f\,g\,dx, the functions 11, cos⁡nπxL\cos\frac{n\pi x}{L} and sin⁡nπxL\sin\frac{n\pi x}{L} are mutually orthogonal, with ⟨1,1⟩=2L\langle 1, 1 \rangle = 2L and squared norm LL for the others, and each coefficient is an orthogonal projection. So the partial sum SNS_N through n=Nn = N is the trigonometric polynomial of that degree closest to ff in mean square. An even ff has every bn=0b_n = 0 and an odd ff has every an=0a_n = 0. For square-integrable ff, Parseval’s identity is

1L∫−LLf(x)2 dx=a022+∑n=1∞(an2+bn2).\begin{aligned} &\frac{1}{L}\int_{-L}^{L} f(x)^2\,dx = \frac{a_0^2}{2} \\[0.4em] &\quad + \sum_{n=1}^{\infty}\left(a_n^2 + b_n^2\right). \end{aligned}

The sawtooth of period 2π2\pi equal to xx on (−π,π)(-\pi, \pi) is odd, with bn=2(−1)n+1nb_n = \frac{2(-1)^{n+1}}{n}, so

x∼2∑n=1∞(−1)n+1nsin⁡nx,x \sim 2\sum_{n=1}^{\infty}\frac{(-1)^{n+1}}{n}\sin nx,

and Parseval’s identity gives ∑n≥11n2=π26\sum_{n \ge 1}\frac{1}{n^2} = \frac{\pi^2}{6}.

Lessons Fourier series

Convergence, Gibbs phenomenon and half-range series

  • Pointwise convergence. Let ff be periodic and piecewise smooth, meaning that in each period ff and f′f' are continuous except at finitely many points, where they have finite one-sided limits. Then the Fourier series converges at every xx to 12(f(x−)+f(x+))\frac{1}{2}\big(f(x^-) + f(x^+)\big), which is f(x)f(x) wherever ff is continuous.
  • Gibbs phenomenon. Near a jump of size JJ the partial sums overshoot on each side by about 0.0895 J0.0895\,J however many terms are taken. The overshoot gets narrower but not lower. The exact fraction is 1π∫0πsin⁡uu du−12≈0.08949\frac{1}{\pi}\int_0^\pi\frac{\sin u}{u}\,du - \frac{1}{2} \approx 0.08949.
  • Decay. Coefficients of a function whose periodic extension has a jump decrease like 1/n1/n, and those of a function with a continuous periodic extension and a piecewise smooth derivative decrease at least like 1/n21/n^2.
  • Termwise operations. Termwise integration is valid for piecewise continuous ff. Termwise differentiation is valid when the periodic extension of ff is continuous and f′f' is piecewise smooth. Differentiating the sawtooth series term by term gives 2∑(−1)n+1cos⁡nx2\sum(-1)^{n+1}\cos nx, whose terms do not even tend to 00.

A function given only on (0,L)(0, L) has a half-range sine series from its odd extension and a half-range cosine series from its even extension,

bn=2L∫0Lf(x)sin⁡nπxL dx,an=2L∫0Lf(x)cos⁡nπxL dx.\begin{aligned} b_n &= \frac{2}{L}\int_0^L f(x)\sin\frac{n\pi x}{L}\,dx, \\[0.4em] a_n &= \frac{2}{L}\int_0^L f(x)\cos\frac{n\pi x}{L}\,dx. \end{aligned}

Sine series suit Dirichlet conditions and cosine series suit Neumann conditions. The constant 11 on (0,π)(0, \pi) has the sine series 4π(sin⁡x+sin⁡3x3+sin⁡5x5+⋯)\frac{4}{\pi}\big(\sin x + \frac{\sin 3x}{3} + \frac{\sin 5x}{5} + \cdots\big), which is 00 at x=0x = 0 and x=πx = \pi, the jump averages of the odd extension, while its cosine series is just 11.

Lessons Convergence and half-range series

Classification and well-posedness

A second-order equation in two variables,

Auxx+2Buxy+Cuyy+(lower order)=f,\begin{aligned} &Au_{xx} + 2Bu_{xy} + Cu_{yy} \\[0.4em] &\quad + (\text{lower order}) = f, \end{aligned}

with AA, BB and CC not all zero at a point, is classified there by the sign of B2−ACB^2 - AC, which comes from the principal part, the second-order terms.

B2−ACB^2 - ACTypeExample
negativeellipticLaplace’s equation uxx+uyy=0u_{xx} + u_{yy} = 0
zeroparabolicthe heat equation ut=κuxxu_t = \kappa u_{xx}
positivehyperbolicthe wave equation utt=c2uxxu_{tt} = c^2u_{xx}

The type can change with position. The equation uxx+y uyy=0u_{xx} + y\,u_{yy} = 0 is elliptic for y>0y > 0 and hyperbolic for y<0y < 0. A problem is well posed when a solution exists, it is unique and it depends continuously on the data, each measured in a stated norm. Data must also be compatible, so for a classical heat solution that is continuous up to the corners, the initial temperature at an end must equal the boundary temperature there at t=0t = 0. The backward heat equation ut=−uxxu_t = -u_{xx} on [0,π][0, \pi] with zero end values is not well posed. Its solutions e−n+n2tsin⁡nxe^{-n + n^2t}\sin nx start with size e−n→0e^{-n} \to 0 but have size en2−n→∞e^{n^2 - n} \to \infty at t=1t = 1.

Lessons Types and well-posedness

Transport and characteristics

The transport equation ut+c ux=0u_t + c\,u_x = 0 with constant speed cc keeps uu constant along the characteristic lines x=ξ+ctx = \xi + ct, so the data u(x,0)=f(x)u(x, 0) = f(x) give u(x,t)=f(x−ct)u(x, t) = f(x - ct), a shape moving at speed cc. With a variable speed, ut+a(x,t) ux=0u_t + a(x, t)\,u_x = 0 keeps uu constant along the characteristic curves dxdt=a(x,t)\frac{dx}{dt} = a(x, t), and with a source term gg the value changes along them by dudt=g\frac{du}{dt} = g. For ut+x ux=0u_t + x\,u_x = 0 the characteristics are x=ξetx = \xi e^t and u=f(xe−t)u = f(xe^{-t}).

On x≥0x \ge 0 with c>0c > 0, information enters at x=0x = 0, the inflow boundary, so boundary data u(0,t)=g(t)u(0, t) = g(t) are given there and nothing is prescribed at an outflow boundary. The solution is f(x−ct)f(x - ct) for x≥ctx \ge ct and g(t−x/c)g(t - x/c) for 0≤x<ct0 \le x < ct, and it is continuous when g(0)=f(0)g(0) = f(0).

For the inviscid Burgers’ equation ut+u ux=0u_t + u\,u_x = 0 with u(x,0)=f(x)u(x, 0) = f(x), each characteristic x=ξ+f(ξ) tx = \xi + f(\xi)\,t carries the value f(ξ)f(\xi). If f′f' is bounded below and negative somewhere, characteristics first cross at the time t∗=−1/inf⁡f′t_* = -1/\inf f', where the smooth solution breaks down. The data f=−xf = -x give u=−x1−tu = -\frac{x}{1 - t} for t<1t < 1. After a breakdown one uses weak solutions, which can jump, together with an admissibility condition that selects the physical one. For Burgers’ equation a jump from uLu_L down to uR<uLu_R < u_L moves at the shock speed uL+uR2\frac{u_L + u_R}{2}.

Lessons Transport and characteristics

Heat equation

The heat equation ut=κ uxxu_t = \kappa\,u_{xx} on 0<x<L0 < x < L, with thermal diffusivity κ>0\kappa > 0 and initial temperature u(x,0)=f(x)u(x, 0) = f(x), is solved by separation of variables. Products u=X(x) T(t)u = X(x)\,T(t) need X′′+λX=0X'' + \lambda X = 0 with the boundary conditions and T′=−κλTT' = -\kappa\lambda T.

EndsSolution
held at 00, u(0,t)=u(L,t)=0u(0, t) = u(L, t) = 0∑n=1∞bne−κ(nπ/L)2tsin⁡nπxL\displaystyle\sum_{n=1}^{\infty} b_ne^{-\kappa(n\pi/L)^2t}\sin\frac{n\pi x}{L}
insulated, ux(0,t)=ux(L,t)=0u_x(0, t) = u_x(L, t) = 0a02+∑n=1∞ane−κ(nπ/L)2tcos⁡nπxL\displaystyle\frac{a_0}{2} + \sum_{n=1}^{\infty} a_ne^{-\kappa(n\pi/L)^2t}\cos\frac{n\pi x}{L}

Here bnb_n and ana_n are the half-range sine and cosine coefficients of ff. Each mode decays at the rate κ(nπ/L)2\kappa(n\pi/L)^2, so high frequencies die fastest and the solution is smooth for t>0t > 0 even when ff is not. Insulated ends keep the mean temperature 1L∫0Lu dx=a02\frac{1}{L}\int_0^L u\,dx = \frac{a_0}{2} constant, and uu tends to it. For fixed end temperatures u(0,t)=Au(0, t) = A and u(L,t)=Bu(L, t) = B, subtract the steady state A+(B−A)x/LA + (B - A)x/L, which leaves a problem with zero end values. With zero end values the quantity 12∫0Lu2 dx\frac{1}{2}\int_0^L u^2\,dx decreases at the rate κ∫0Lux2 dx\kappa\int_0^L u_x^2\,dx. On [0,π][0, \pi] with κ=2\kappa = 2, the data f=sin⁡x+2sin⁡2xf = \sin x + 2\sin 2x and zero end values give u=e−2tsin⁡x+2e−8tsin⁡2xu = e^{-2t}\sin x + 2e^{-8t}\sin 2x.

Lessons Heat equation

Wave equation

The wave equation utt=c2uxxu_{tt} = c^2u_{xx} needs an initial displacement u(x,0)=f(x)u(x, 0) = f(x) and an initial velocity ut(x,0)=g(x)u_t(x, 0) = g(x). With fixed ends u(0,t)=u(L,t)=0u(0, t) = u(L, t) = 0 the normal modes sin⁡nπxL\sin\frac{n\pi x}{L} vibrate at the frequencies ωn=nπcL\omega_n = \frac{n\pi c}{L}, and

u=∑n=1∞Tn(t)sin⁡nπxL,Tn=fncos⁡ωnt+gnωnsin⁡ωnt,\begin{gathered} u = \sum_{n=1}^{\infty} T_n(t)\sin\frac{n\pi x}{L}, \\[0.4em] T_n = f_n\cos\omega_nt + \frac{g_n}{\omega_n}\sin\omega_nt, \end{gathered}

where fnf_n and gng_n are the sine coefficients of ff and gg. Free ends, ux=0u_x = 0, give cosine modes and a zero mode A+BtA + Bt that moves the whole string. On the whole line, d’Alembert’s formula is

u(x,t)=f(x−ct)2+f(x+ct)2+12c∫x−ctx+ctg(s) ds.\begin{aligned} &u(x, t) = \frac{f(x - ct)}{2} \\[0.4em] &\quad + \frac{f(x + ct)}{2} \\[0.4em] &\quad + \frac{1}{2c}\int_{x - ct}^{x + ct} g(s)\,ds. \end{aligned}

So u(x,t)u(x, t) depends only on the data in the domain of dependence [x−ct,x+ct][x - ct, x + ct], and disturbances travel no faster than cc. Every twice differentiable solution on the whole line has the form F(x−ct)+G(x+ct)F(x - ct) + G(x + ct), two traveling waves. With fixed or free ends the energy 12∫0L(ut2+c2ux2) dx\frac{1}{2}\int_0^L (u_t^2 + c^2u_x^2)\,dx is constant. On [0,π][0, \pi] with c=2c = 2, the data f=sin⁡xf = \sin x and g=sin⁡2xg = \sin 2x give u=sin⁡xcos⁡2t+14sin⁡2xsin⁡4tu = \sin x\cos 2t + \frac{1}{4}\sin 2x\sin 4t, with energy 5π4\frac{5\pi}{4}.

Lessons Wave equation

Laplace and Poisson equations

Write Δu=uxx+uyy\Delta u = u_{xx} + u_{yy}, which in polar coordinates is urr+1rur+1r2uθθu_{rr} + \frac{1}{r}u_r + \frac{1}{r^2}u_{\theta\theta}. A harmonic function solves Laplace’s equation Δu=0\Delta u = 0, and Poisson’s equation is Δu=f\Delta u = f.

  • Rectangle. On 0<x<a0 < x < a, 0<y<b0 < y < b with u=0u = 0 on three sides and u(x,b)=g(x)u(x, b) = g(x), separation of variables gives u=∑cnsin⁡nπxasinh⁡nπyau = \sum c_n\sin\frac{n\pi x}{a}\sinh\frac{n\pi y}{a}, where cnsinh⁡nπbac_n\sinh\frac{n\pi b}{a} is the nnth half-range sine coefficient of gg.
  • Disk. On r<Rr < R with u(R,θ)=g(θ)u(R, \theta) = g(\theta), the bounded solution is u=a02+∑n≥1(rR)n(ancos⁡nθ+bnsin⁡nθ)u = \frac{a_0}{2} + \sum_{n \ge 1}\left(\frac{r}{R}\right)^n(a_n\cos n\theta + b_n\sin n\theta), with the Fourier coefficients of gg. The separated solutions r−nr^{-n} and ln⁡r\ln r are excluded because they are unbounded at the center.
  • Maximum principle. A function harmonic in a bounded region and continuous up to its boundary takes its maximum and its minimum on the boundary. So the Dirichlet problem, with uu given on the boundary, has at most one solution.
  • Neumann problem. For Δu=f\Delta u = f in a region Ω\Omega with normal derivative ∂u/∂n=h\partial u/\partial n = h on its boundary, a solution can exist only if ∫Ωf dA=∮∂Ωh ds\int_\Omega f\,dA = \oint_{\partial\Omega} h\,ds, and it is then unique up to an added constant.

The function x2−y2=r2cos⁡2θx^2 - y^2 = r^2\cos 2\theta is harmonic. The function u=x2+y2−1u = x^2 + y^2 - 1 solves Δu=4\Delta u = 4 in the unit disk with u=0u = 0 on the circle. Its normal derivative on the circle is 22, and the compatibility condition checks, since 4π=2⋅2π4\pi = 2 \cdot 2\pi.

Lessons Laplace and Poisson equations

Fourier transform

For ff integrable on the real line R\mathbb{R}, the Fourier transform and its inverse are

f^(k)=∫Rf(x) e−ikx dx,f(x)=12π∫Rf^(k) eikx dk,\begin{aligned} \hat{f}(k) &= \int_{\mathbb{R}} f(x)\,e^{-ikx}\,dx, \\[0.4em] f(x) &= \frac{1}{2\pi}\int_{\mathbb{R}}\hat{f}(k)\,e^{ikx}\,dk, \end{aligned}

where the inverse formula holds at every point where ff is continuous when f^\hat{f} is integrable too.

FunctionFourier transformConditions
f′(x)f'(x)ik f^(k)ik\,\hat{f}(k)ff and f′f' integrable
f′′(x)f''(x)−k2f^(k)-k^2\hat{f}(k)ff, f′f' and f′′f'' integrable
f(x−c)f(x - c)e−ikcf^(k)e^{-ikc}\hat{f}(k)cc real
∫Rf(x−y) g(y) dy\displaystyle\int_{\mathbb{R}} f(x - y)\,g(y)\,dyf^(k) g^(k)\hat{f}(k)\,\hat{g}(k)ff and gg integrable
e−ax2e^{-ax^2}πa e−k2/(4a)\sqrt{\dfrac{\pi}{a}}\,e^{-k^2/(4a)}a>0a > 0

For the heat equation ut=κ uxxu_t = \kappa\,u_{xx} on the whole line with u(x,0)=f(x)u(x, 0) = f(x), taking the Fourier transform in xx gives u^t=−κk2u^\hat{u}_t = -\kappa k^2\hat{u}, so u^(k,t)=e−κk2tf^(k)\hat{u}(k, t) = e^{-\kappa k^2t}\hat{f}(k). For bounded continuous ff and t>0t > 0,

u(x,t)=∫RK(x−y,t) f(y) dy,K(x,t)=14πκt e−x2/(4κt).\begin{aligned} &u(x, t) \\[0.4em] &\quad = \int_{\mathbb{R}} K(x - y, t)\,f(y)\,dy, \\[0.4em] &K(x, t) = \frac{1}{\sqrt{4\pi\kappa t}}\,e^{-x^2/(4\kappa t)}. \end{aligned}

The heat kernel KK is positive with integral 11, so for integrable ff the total heat ∫u dx\int u\,dx stays constant. With κ=1\kappa = 1 the data e−x2e^{-x^2} give u=11+4t e−x2/(1+4t)u = \frac{1}{\sqrt{1 + 4t}}\,e^{-x^2/(1 + 4t)}, whose integral stays π\sqrt{\pi}. Backward in time each frequency would grow like eκk2te^{\kappa k^2t}, which is why the backward problem is not well posed.

Lessons Fourier transforms on unbounded domains

Existence Constructions and Advanced Extensions

The course closes with a constructive proof of existence and three introductions to further study. Picard iteration builds the solution of an initial value problem as a limit, perturbation methods approximate solutions when a parameter is small, and delay and differential-algebraic equations show what changes when the state includes the past or must satisfy constraints.

Picard iteration

For y′=f(t,y)y' = f(t, y), y(t0)=y0y(t_0) = y_0 with ff continuous, a continuous function yy solves the problem on an interval exactly when it solves the integral equation

y(t)=y0+∫t0tf(s,y(s)) ds.\begin{aligned} &y(t) = y_0 \\[0.4em] &\quad + \int_{t_0}^{t} f(s, y(s))\,ds. \end{aligned}

Picard iteration starts from the constant φ0(t)=y0\varphi_0(t) = y_0 and repeats

φn+1(t)=y0+∫t0tf(s,φn(s)) ds.\begin{aligned} &\varphi_{n+1}(t) = y_0 \\[0.4em] &\quad + \int_{t_0}^{t} f(s, \varphi_n(s))\,ds. \end{aligned}

Picard-Lindelöf theorem. Let ff be continuous on the rectangle R={∣t−t0∣≤a, ∣y−y0∣≤b}R = \{\lvert t - t_0 \rvert \le a,\ \lvert y - y_0 \rvert \le b\} with ∣f∣≤M\lvert f \rvert \le M and ∣f(t,y1)−f(t,y2)∣≤L ∣y1−y2∣\lvert f(t, y_1) - f(t, y_2) \rvert \le L\,\lvert y_1 - y_2 \rvert on RR. Then on ∣t−t0∣≤h\lvert t - t_0 \rvert \le h with h=min⁡(a,b/M)h = \min(a, b/M), the iterates converge uniformly to the unique solution. On an interval ∣t−t0∣≤T\lvert t - t_0 \rvert \le T with MT≤bMT \le b and q=LT<1q = LT < 1, the iteration is a contraction in the sup norm, and

∥φn−y∥∞≤qn1−q ∥φ1−φ0∥∞.\begin{aligned} &\lVert \varphi_n - y \rVert_\infty \\[0.4em] &\quad \le \frac{q^n}{1 - q}\,\lVert \varphi_1 - \varphi_0 \rVert_\infty. \end{aligned}

The condition MT≤bMT \le b keeps the iterates inside RR, and LT<1LT < 1 makes them contract. Existence is local, as y′=y2y' = y^2 with y(0)=1y(0) = 1 shows, since its solution 11−t\frac{1}{1 - t} exists only for t<1t < 1. For y′=1+yy' = 1 + y with y(0)=0y(0) = 0, the iterates are φ1=t\varphi_1 = t, φ2=t+t22\varphi_2 = t + \frac{t^2}{2} and φ3=t+t22+t36\varphi_3 = t + \frac{t^2}{2} + \frac{t^3}{6}, partial sums of the solution et−1e^t - 1. On 0≤t≤120 \le t \le \frac{1}{2} with b=1b = 1 the constants are M=2M = 2 and L=1L = 1, so q=12q = \frac{1}{2} and the bound gives ∥φ3−y∥∞≤18\lVert \varphi_3 - y \rVert_\infty \le \frac{1}{8}, while the actual error is about 0.0030.003.

Lessons Picard iteration

Perturbation methods

These methods approximate a problem with a small parameter 0<ε≪10 < \varepsilon \ll 1, and each approximation comes with an interval of validity that should be stated.

  • Regular perturbation. Substitute y=y0+εy1+ε2y2+⋯y = y_0 + \varepsilon y_1 + \varepsilon^2y_2 + \cdots and collect equal powers of ε\varepsilon. Each yky_k solves a linear problem.
  • Secular terms. For y′′+(1+ε)2y=0y'' + (1 + \varepsilon)^2y = 0 with y(0)=1y(0) = 1 and y′(0)=0y'(0) = 0, the exact solution cos⁡((1+ε)t)\cos\big((1 + \varepsilon)t\big) has the expansion cos⁡t−εtsin⁡t+⋯\cos t - \varepsilon t\sin t + \cdots, whose term εtsin⁡t\varepsilon t\sin t grows with tt. The expansion is accurate only while εt\varepsilon t is small.
  • Multiple scales. A slow time T=εtT = \varepsilon t lets the amplitudes vary, and removing the secular terms gives y≈cos⁡(t+T)y \approx \cos(t + T), which here is exact.
  • Singular perturbation. When ε\varepsilon multiplies the highest derivative, setting ε=0\varepsilon = 0 lowers the order and the reduced problem cannot meet every boundary condition, so a thin boundary layer forms.

For εy′′+y′=0\varepsilon y'' + y' = 0 in the variable xx with y(0)=0y(0) = 0 and y(1)=1y(1) = 1, the outer solution is y=1y = 1, the inner variable ξ=x/ε\xi = x/\varepsilon gives the layer 1−e−ξ1 - e^{-\xi}, and the composite approximation 1−e−x/ε1 - e^{-x/\varepsilon} differs from the exact solution 1−e−x/ε1−e−1/ε\frac{1 - e^{-x/\varepsilon}}{1 - e^{-1/\varepsilon}} by at most e−1/εe^{-1/\varepsilon}.

Lessons Perturbation methods and multiple scales

Delay differential equations

A delay equation such as y′(t)=F(t,y(t),y(t−τ))y'(t) = F\big(t, y(t), y(t - \tau)\big) with τ>0\tau > 0 needs as initial data a history function φ\varphi on [t0−τ,t0][t_0 - \tau, t_0], not just the value y(t0)y(t_0). Its state at time tt is the whole piece of yy on [t−τ,t][t - \tau, t], so the state space is infinite-dimensional. In the method of steps, the delayed values on [t0,t0+τ][t_0, t_0 + \tau] come from φ\varphi, so the equation there is an ordinary one, and its solution supplies the delayed values for the next interval of length τ\tau. A jump in a derivative at t0t_0 reappears one delay later in a higher derivative.

For y′(t)=−y(t−1)y'(t) = -y(t - 1) with history 11 on [−1,0][-1, 0], the method gives y=1−ty = 1 - t on [0,1][0, 1] and y=t22−2t+32y = \frac{t^2}{2} - 2t + \frac{3}{2} on [1,2][1, 2]. The history 1+s1 + s has the same value y(0)=1y(0) = 1 but gives y=1−t22y = 1 - \frac{t^2}{2} on [0,1][0, 1], a different future.

For y′(t)=−a y(t−τ)y'(t) = -a\,y(t - \tau) with a>0a > 0 and τ>0\tau > 0, the function eλte^{\lambda t} is a solution when

λ=−a e−λτ,\lambda = -a\,e^{-\lambda\tau},

an equation with infinitely many complex roots. Every root has negative real part, and the zero solution is asymptotically stable, exactly when aτ<π2a\tau < \frac{\pi}{2}. At aτ=π2a\tau = \frac{\pi}{2} the roots ±ia\pm ia cross the imaginary axis, so a long enough delay destabilizes feedback that is stable without delay.

Lessons Delay differential equations

Differential-algebraic equations

A semi-explicit differential-algebraic equation x′=f(t,x,z)x' = f(t, x, z), 0=g(t,x,z)0 = g(t, x, z) has differential variables xx and algebraic variables zz, which are fixed at each instant by the constraint rather than by their own rate equations. Initial values must be consistent, satisfying the constraint and every hidden constraint found by differentiating it. The differentiation index is the number of times the constraints must be differentiated to obtain an ordinary differential equation for all the variables. If ∂g/∂z\partial g/\partial z is invertible, the constraint determines zz locally and one differentiation gives z′z', so the index is 11.

For x′=−x+zx' = -x + z and 0=x+z−10 = x + z - 1, the value x(0)=0x(0) = 0 forces z(0)=1z(0) = 1, and z=1−xz = 1 - x gives x′=1−2xx' = 1 - 2x, so x=12(1−e−2t)x = \frac{1}{2}(1 - e^{-2t}) and z=12(1+e−2t)z = \frac{1}{2}(1 + e^{-2t}). A pendulum with unit mass, length and gravity, with yy measured upward, written as x′=vx' = v, y′=wy' = w, v′=−λxv' = -\lambda x, w′=−1−λyw' = -1 - \lambda y with the constraint x2+y2=1x^2 + y^2 = 1, needs three differentiations. The first gives xv+yw=0xv + yw = 0, so the velocity is tangent to the circle, the second gives the multiplier λ=v2+w2−y\lambda = v^2 + w^2 - y, and the third gives λ′\lambda', so this formulation has index 33. A differentiated constraint alone loses information, since x′+z′=0x' + z' = 0 only says that x+zx + z is constant, and consistent initial values select the constant 11.

Lessons Differential-algebraic equations