Convolution and Linear-System Response, differential equations lesson

Dye trickles into a tank while an outlet slowly washes it out. The amount in the tank now depends on everything that has flowed in so far, each earlier portion weighted by how much of it is still left. Convolution writes that bookkeeping as one integral over the past. This lesson defines causal convolution, shows why the Laplace transform turns it into a product, computes a zero-state response in two independent ways and rewrites the same problem as an integral equation. Throughout, $t \ge 0$ is time and a prime means a derivative with respect to $t$ .

Key terms

Causal convolution

Definition Let $f$ and $g$ be piecewise continuous on $[0, \infty)$ . Their causal convolution is the function defined for $t \ge 0$ by

\begin{aligned} &(f * g)(t) \\[0.4em] &\quad = \int_0^t f(t - \tau)\,g(\tau)\,d\tau. \end{aligned}

Only the values of $g$ on $[0, t]$ enter, so the convolution at time $t$ never uses the future. It is the convolution over the whole real line of the two functions after both are set to zero for negative times, which is what causal means here. The substitution $\sigma = t - \tau$ shows that $f * g = g * f$ , and the operation is linear in each factor. It is not the pointwise product $f(t)g(t)$ .

In plain words To find the effect at time $t$ , take every earlier time $\tau$ , weight the input $g(\tau)$ by the value of $f$ at the elapsed time $t - \tau$ , and add up the contributions from time $0$ to now.

Example For $f(t) = t$ and $g(t) = 1$ ,

\begin{aligned} (f * g)(t) &= \int_0^t (t - \tau)\,d\tau \\[0.4em] &= \frac{t^2}{2}, \end{aligned}

and the other order gives $\int_0^t 1 \cdot \tau\,d\tau = t^2/2$ as well. With $f = g = 1$ the convolution is $\int_0^t d\tau = t$ , while the pointwise product is $1$ . Convolution builds a new function of time out of the whole past, not a product of current values.

Convolution theorem

Definition Let $f$ and $g$ be piecewise continuous on $[0, \infty)$ and of exponential order $a$ , with Laplace transforms $F$ and $G$ . Then for every complex $s$ with $\operatorname{Re} s > a$ the Laplace transform of $f * g$ exists and

\mathcal{L}\{f * g\}(s) = F(s)\,G(s).

To see why, write $\mathcal{L}\{f * g\}(s)$ as a double integral over the region $0 \le \tau \le t$ and replace the observation time $t$ by $\tau + \sigma$ , where $\sigma = t - \tau \ge 0$ is the elapsed time. The weight splits as $e^{-st} = e^{-s\tau}e^{-s\sigma}$ , and the double integral separates into the integral of $e^{-s\tau}g(\tau)$ times the integral of $e^{-s\sigma}f(\sigma)$ , each over $[0, \infty)$ . Changing the order of integration is allowed because for $\operatorname{Re} s > a$ the double integral of the absolute values is finite. Read backward, the theorem says $\mathcal{L}^{-1}\{FG\} = f * g$ , which inverts a product without partial fractions. There is no such rule for the pointwise product, and $\mathcal{L}\{fg\}$ is in general not $FG$ .

In plain words Adding up weighted past inputs in time becomes plain multiplication in the variable $s$ . A response that needs an integral in time needs only a product after the Laplace transform.

Example For $f(t) = e^{t}$ and $g(t) = e^{2t}$ ,

\begin{aligned} (f * g)(t) &= \int_0^t e^{t - \tau}e^{2\tau}\,d\tau \\[0.4em] &= e^{t}\left(e^{t} - 1\right) \\[0.4em] &= e^{2t} - e^{t}. \end{aligned}

Both functions are of exponential order $2$ . For $\operatorname{Re} s > 2$ the Laplace transform of the result is

\begin{aligned} &\frac{1}{s - 2} - \frac{1}{s - 1} \\[0.4em] &\quad = \frac{1}{(s - 1)(s - 2)}, \end{aligned}

which is exactly the product of $F(s) = 1/(s - 1)$ and $G(s) = 1/(s - 2)$ .

Impulse-response kernel

Definition Consider a linear equation with constant coefficients,

\begin{aligned} &a_n y^{(n)} + \cdots + a_1 y' + a_0 y \\[0.4em] &\quad = f(t), \qquad a_n \ne 0, \end{aligned}

with input $f$ and characteristic polynomial $p(s) = a_n s^n + \cdots + a_1 s + a_0$ . Its impulse-response kernel is the function $h$ on $[0, \infty)$ whose Laplace transform is

H(s) = \frac{1}{p(s)}.

Equivalently, $h$ is the solution of the homogeneous equation with $h(0) = h'(0) = \cdots = h^{(n-2)}(0) = 0$ and $h^{(n-1)}(0) = 1/a_n$ . For a first-order equation this simply means $h(0) = 1/a_1$ . The kernel depends only on the equation, not on the input. It is the response to a unit impulse at time $0$ of the system at rest just before that instant, which explains its name. Inside a convolution, $h(t - \tau)$ is the weight that an input arriving at time $\tau$ still carries at the later time $t$ .

In plain words The kernel is the memory of the system. Its value at the elapsed time $t - \tau$ says how much of one unit of input delivered at time $\tau$ is still felt at time $t$ .

Example For $y' + 2y = f$ the characteristic polynomial is $s + 2$ , so $h(t) = e^{-2t}$ , and indeed $h(0) = 1$ . Input that arrived one time unit ago still counts with the weight $e^{-2} \approx 0.135$ , and input from two units ago with $e^{-4} \approx 0.018$ , so this system forgets quickly. For $y'' + y = f$ the kernel is $h(t) = \sin t$ , with $h(0) = 0$ and $h'(0) = 1$ . These weights never fade, and an undamped oscillator keeps every past input in its motion.

Zero-state response

Definition For the equation above, the zero-state response to an input $f$ is the solution with zero initial values, $y(0) = y'(0) = \cdots = y^{(n-1)}(0) = 0$ . The system starts at rest, so everything it does is caused by the input. For every $f$ that is piecewise continuous on $[0, \infty)$ ,

\begin{aligned} &y_{\mathrm{zs}}(t) = (h * f)(t) \\[0.4em] &\quad = \int_0^t h(t - \tau)\,f(\tau)\,d\tau. \end{aligned}

When $f$ is of exponential order, the Laplace transform of the equation with zero initial values is $p(s)Y = F$ , so $Y = HF$ , and the convolution theorem gives the formula. For a general piecewise continuous $f$ it still holds, because neither side at time $t$ depends on the input after time $t$ . With nonzero initial values the solution is $y = y_{\mathrm{zi}} + y_{\mathrm{zs}}$ , where the zero-input response $y_{\mathrm{zi}}$ solves the homogeneous equation with the given initial values.

In plain words Start the system at rest, feed it the input, and add up the weighted past. Whatever the system would do on its own from a nonzero start is a separate term.

Example Take $y' + 2y = 3$ with $y(0) = 0$ , so $h(t) = e^{-2t}$ and $f = 3$ . The convolution is

\begin{aligned} y(t) &= \int_0^t 3e^{-2(t - \tau)}\,d\tau \\[0.4em] &= \tfrac{3}{2}\left(1 - e^{-2t}\right). \end{aligned}

The Laplace route agrees. Here $H(s) = 1/(s + 2)$ and $F(s) = 3/s$ , and partial fractions give

\begin{aligned} Y(s) &= \frac{3}{s(s + 2)} \\[0.4em] &= \frac{3/2}{s} - \frac{3/2}{s + 2}. \end{aligned}

Substituting back, $y' = 3e^{-2t}$ and $y' + 2y = 3$ , with $y(0) = 0$ . The response levels off at $\tfrac{3}{2}$ , where $2y = 3$ balances the input. If instead $y(0) = 1$ , the zero-input response $e^{-2t}$ is added, and the solution becomes $y = \tfrac{3}{2} - \tfrac{1}{2}e^{-2t}$ .

Volterra integral equation

Definition A linear Volterra integral equation of the second kind for an unknown function $y$ on $[0, \infty)$ has the form

\begin{aligned} &y(t) = g(t) \\[0.4em] &\quad + \int_0^t k(t, \tau)\,y(\tau)\,d\tau, \end{aligned}

with a given function $g$ and a given kernel $k$ . The upper limit is the current time, so the present value of $y$ is fixed by its own past. If $g$ is continuous on $[0, \infty)$ and $k$ is continuous for $0 \le \tau \le t$ , the equation has exactly one continuous solution. When the kernel depends only on the elapsed time, $k(t, \tau) = k(t - \tau)$ , the integral is the causal convolution $k * y$ . The convolution theorem then turns the equation into $Y = G + KY$ , so $Y = G/(1 - K)$ wherever these Laplace transforms exist and $K(s) \ne 1$ . Integrating an initial-value problem from $0$ to $t$ produces such an equation, with the initial values built into $g$ .

In plain words Instead of saying how fast $y$ changes, a Volterra equation says what $y$ is now in terms of everything it has been. A derivative looks at the present instant, while the integral looks back over the whole past.

Example Integrate $y' + 2y = 3$ from $0$ to $t$ . The first term gives $y(t) - y(0)$ , which is $y(t)$ when $y(0) = 0$ , so

y(t) = 3t - 2\int_0^t y(\tau)\,d\tau.

Here $g(t) = 3t$ and $k = -2$ . Taking Laplace transforms gives $Y = 3/s^2 - 2Y/s$ , so $(1 + 2/s)Y = 3/s^2$ and $Y = 3/(s(s + 2))$ , the same as before. Conversely, if $y$ is a continuous solution, the right side is differentiable, so $y$ is too. Differentiating gives $y' = 3 - 2y$ , and setting $t = 0$ gives $y(0) = 0$ . The integral equation therefore carries both the differential equation and its initial value, and its solution is again $\tfrac{3}{2}(1 - e^{-2t})$ .

Check yourself

Find the zero-state response of $y'' + y = 1$ by convolution, and check it.

Show the answer

The kernel is $h(t) = \sin t$ and the input is $f = 1$ , so

\begin{aligned} y(t) &= \int_0^t \sin(t - \tau)\,d\tau \\[0.4em] &= \Bigl[\cos(t - \tau)\Bigr]_{\tau = 0}^{\tau = t} \\[0.4em] &= 1 - \cos t. \end{aligned}

As a check, $y' = \sin t$ and $y'' = \cos t$ , so $y'' + y = \cos t + 1 - \cos t = 1$ , with $y(0) = 0$ and $y'(0) = 0$ . The Laplace route gives the same result, since $\frac{1}{s^2 + 1} \cdot \frac{1}{s} = \frac{1}{s} - \frac{s}{s^2 + 1}$ .

A classmate claims that $\mathcal{L}\{t\,e^{-t}\}$ equals $1/(s^2(s + 1))$ , because $\mathcal{L}\{t\} = 1/s^2$ and $\mathcal{L}\{e^{-t}\} = 1/(s + 1)$ . What is wrong, and which function does have this Laplace transform?

Show the answer

A product of Laplace transforms belongs to the convolution, not to the pointwise product. Directly from the definition, $\mathcal{L}\{t\,e^{-t}\}(s) = \int_0^\infty t\,e^{-(s + 1)t}\,dt$ is $\mathcal{L}\{t\}$ evaluated at $s + 1$ , so

\begin{gathered} \mathcal{L}\{t\,e^{-t}\} = \frac{1}{(s + 1)^2}, \\[0.4em] \operatorname{Re} s > -1. \end{gathered}

The product $1/(s^2(s + 1))$ is the Laplace transform of the convolution of $t$ with $e^{-t}$ ,

\begin{aligned} &\int_0^t (t - \tau)\,e^{-\tau}\,d\tau \\[0.4em] &\quad = t - 1 + e^{-t}, \end{aligned}

in agreement with the partial fractions $-1/s + 1/s^2 + 1/(s + 1)$ . The two candidates really differ. At $s = 1$ , for example, they equal $\tfrac{1}{4}$ and $\tfrac{1}{2}$ .

Solve the Volterra integral equation $y(t) = 1 + \int_0^t (t - \tau)\,y(\tau)\,d\tau$ for $t \ge 0$ .

Show the answer

The kernel $t - \tau$ depends only on the elapsed time, so the integral is the convolution of $t$ with $y$ . With $\mathcal{L}\{t\} = 1/s^2$ the Laplace transform of the equation is $Y = 1/s + Y/s^2$ , so $(1 - 1/s^2)Y = 1/s$ and

Y(s) = \frac{s}{s^2 - 1},

which is the Laplace transform of $\cosh t$ for $\operatorname{Re} s > 1$ . So $y(t) = \cosh t$ . As a check, integration by parts gives $\int_0^t (t - \tau)\cosh\tau\,d\tau = \cosh t - 1$ , so the right side equals $1 + \cosh t - 1 = \cosh t$ . Differentiating the equation twice gives the same answer another way, since it yields $y'' = y$ with $y(0) = 1$ and $y'(0) = 0$ .

Convolution and Linear-System Response

Key terms

Causal convolution

Convolution theorem

Impulse-response kernel

Zero-state response

Volterra integral equation

Check yourself

How this lesson connects

Builds on

Used later in