Unit 14, lesson 105

Limit order books and queueing models

How a FIFO queue model of the limit order book gives the probability that a resting order fills by a deadline, using arrival intensities, Poisson counts, Erlang waiting times and birth-death extensions.

  • Video 105
  • Order books, execution and market making

A buy limit order rests at the best bid behind four units of earlier orders. Will it have filled two minutes from now? The displayed depth at its price cannot answer that, because the answer depends on which events move the order forward, how fast they arrive and whether the order and its price level stay in place. This lesson builds a pure queue-depletion model in which the fill probability is an exact Poisson tail and the fill time has an Erlang distribution, and then shows how births ahead of the order and an absorbing fill state change the equations.

Key terms

Limit order book

Definition A limit order book is the record, at a given time, of the resting limit orders for one instrument on one trading venue, grouped by side and price level, with prices in currency units. A buy limit order names the highest price it accepts and a sell limit order the lowest. The best bid is the highest price with resting buy orders and the best ask is the lowest price with resting sell orders. In a continuous book, orders that cross execute at once, so the best bid stays below the best ask. A marketable order, such as a market order or a limit order priced to cross, executes against resting orders on the opposite side, starting at the best price. Queue priority is the rule that decides which resting units at a price level execute first. Under price-time priority better prices execute first, and at one price level orders execute first-in-first-out (FIFO), in order of arrival. The queue position of an order is the number of eligible units ahead of it under the venue’s rule, which need not equal the displayed depth at its level. Other allocation rules such as pro-rata matching, hidden orders, amendments that lose priority and cancellations all change which units are effectively ahead. Under FIFO a new limit order at the same price joins the back of the queue, and a cancellation behind an order leaves its position unchanged.

In plain words The book shows how much is offered at each price, but a resting order is served only after the units ahead of it are gone. For a waiting order the useful number is its place in line, not the size of the crowd at its price.

Example In a constructed book, the bids are 66 units at 99.9899.98 and 99 units at 99.9999.99, and the asks are 55 units at 100.01100.01 and 88 units at 100.02100.02. The best bid is 99.9999.99, the best ask is 100.01100.01, the spread is 0.020.02 and the midpoint is 100.00100.00. Our one-unit buy order rests at 99.9999.99 with 44 units ahead and 44 units behind, so the displayed depth at our level is 99 units while our queue position is 44. A market sell of 33 units executes against the first 33 units at 99.9999.99 and leaves one unit ahead of us. A cancellation behind us changes nothing, while a cancellation of that last unit ahead puts us at the front. A new buy order at 100.00100.00 would become the best bid, and incoming sells would trade against it before reaching our level.

Arrival intensity

Definition Let NtN_t count a specified class of book events up to time tt, for example the units removed from the front of our queue, and let Ft\mathcal{F}_t be the information available at time tt, such as the past of the book. An arrival intensity of NN is a nonnegative process λt\lambda_t, known just before tt, such that Nt−∫0tλs dsN_t - \int_0^t \lambda_s \, ds is a local martingale with respect to Ft\mathcal{F}_t, and a martingale when λ\lambda is bounded. It is measured in events per unit of time. Under mild regularity, for example when λ\lambda is bounded with continuous paths,

λt=lim⁡Δ↓01Δ×E[Nt+Δ−Nt∣Ft],\begin{aligned} \lambda_t &= \lim_{\Delta \downarrow 0} \frac{1}{\Delta} \\[0.4em] &\quad \times \mathbb{E}\bigl[N_{t+\Delta} - N_t \mid \mathcal{F}_t\bigr], \end{aligned}

so the intensity is the conditional expected number of events per unit time, and λtΔ\lambda_t \Delta is the expected count over a short window. A counting process with jumps of size one and a constant intensity is a Poisson process, the subject of the next card. A state-dependent intensity is a function λ(x)\lambda(x) of the current book state xx, such as the sizes of the best queues, as in queue-reactive models. The class of events has to be fixed before its rate means anything. For an order waiting in a queue, executions at its level and cancellations ahead of it move it forward, cancellations behind it and new orders at its price do not, and a cancellation of the order itself ends its chance to fill and is not an execution.

In plain words The intensity is the speed at which the counted events arrive, given what is known now. Multiplied by a short interval it gives the expected number of events in that interval, which for a short enough interval is also close to the probability of an event. It is a rate, so over a long interval the product can exceed one and is then not a probability at all.

Example Suppose market sells of one unit hit our level at the constant intensity μ=3\mu = 3 per minute. Over Δ=0.01\Delta = 0.01 minutes, which is 0.60.6 seconds, the expected count is 0.030.03, and under the Poisson model the probability of at least one event is 1−e−0.03≈0.029551 - e^{-0.03} \approx 0.02955, close to the first-order value 0.030.03. Over one minute the expected count is 33, while the probability of at least one event is 1−e−3≈0.95021 - e^{-3} \approx 0.9502. If cancellations ahead also counted, their rate would shrink as the units ahead disappear and would be zero once we reach the front, so the intensity of our progress would depend on the queue state.

Poisson process

Definition A Poisson process with rate μ>0\mu > 0 is a counting process (Nt)t≥0(N_t)_{t \ge 0} with N0=0N_0 = 0 and independent increments such that, for all t≥0t \ge 0 and h>0h > 0, the count Nt+h−NtN_{t+h} - N_t has the Poisson distribution with mean μh\mu h,

P(Nt+h−Nt=k)=e−μh (μh)kk!\begin{aligned} &\mathbb{P}(N_{t+h} - N_t = k) \\[0.4em] &\quad = e^{-\mu h}\, \frac{(\mu h)^k}{k!} \end{aligned}

for every integer k≥0k \ge 0. Equivalently, the times between consecutive events are independent and exponentially distributed with rate μ\mu, so each has mean 1/μ1/\mu. The intensity of a Poisson process with respect to its own history is the constant μ\mu, all its jumps have size one, and μh\mu h is the dimensionless expected count over a window of length hh. The sum of two independent Poisson processes with rates μ1\mu_1 and μ2\mu_2 is a Poisson process with rate μ1+μ2\mu_1 + \mu_2. A single event that removes several units at once is not a jump of size one, so it needs a different counting model.

In plain words Events arrive at a steady rate with no memory. How many arrived in the last minute says nothing about the next minute, and in every window the count has one fixed law that depends only on the window’s length.

Example With μ=3\mu = 3 eligible removals per minute and a horizon of h=2h = 2 minutes, the expected count is μh=6\mu h = 6. The probabilities of exactly 00, 11, 22, 33 and 44 events are e−6e^{-6} times 11, 66, 1818, 3636 and 5454, which are about 0.002480.00248, 0.014870.01487, 0.044620.04462, 0.089240.08924 and 0.133850.13385. At most four events therefore occur with probability 115 e−6≈0.28506115\, e^{-6} \approx 0.28506.

Fill probability

Definition The fill probability of a resting order over a horizon hh is the probability P(Tfill≤h)\mathbb{P}(T_{\text{fill}} \le h), under the model’s probability measure P\mathbb{P}, that the order is completely executed within time hh, where TfillT_{\text{fill}} is its fill time and Tfill=∞T_{\text{fill}} = \infty if it never fills. The pure queue-depletion model makes these assumptions. The order is one unit with a∈{0,1,2,… }a \in \{0, 1, 2, \dots\} eligible units ahead of it under FIFO priority. Each counted service event removes exactly one unit from the front of the queue, first the units ahead and then ours. The service events form a Poisson process NN with rate μ>0\mu > 0. No unit ever enters ahead of ours, the price level stays available, and we do not cancel. The order then fills at the (a+1)(a+1)-th service event, so {Tfill≤h}={Nh≥a+1}\{T_{\text{fill}} \le h\} = \{N_h \ge a + 1\}, and the complement collects the counts 00 through aa,

P(Tfill≤h)=1−e−μh∑k=0a(μh)kk!.\begin{aligned} &\mathbb{P}(T_{\text{fill}} \le h) \\[0.4em] &\quad = 1 - e^{-\mu h} \sum_{k=0}^{a} \frac{(\mu h)^k}{k!} . \end{aligned}

The probability depends on μ\mu and hh only through m=μhm = \mu h. Writing it as g(m)g(m), the sum telescopes when differentiated, and

g′(m)=e−mmaa!,g'(m) = \frac{e^{-m} m^{a}}{a!},

which is positive for m>0m > 0. So the fill probability is 00 at h=0h = 0, strictly increases with the rate and the horizon, and tends to 11 as h→∞h \to \infty. For h>0h > 0 it strictly decreases with the number of units ahead, because one more unit ahead subtracts P(Nh=a+1)>0\mathbb{P}(N_h = a + 1) > 0. A fill probability says nothing about the quality of the fill. A passive buy order is more likely to fill when sellers are active, and sellers may be trading on information that the price will fall. Expected profit given a fill therefore depends on the joint law of fills and later prices, together with fees and inventory, and simulating fills independently of adverse price moves can overstate the value of a strategy even when the average fill rate is right.

In plain words In the pure model the order fills by the deadline exactly when the number of service events before the deadline exceeds the number of units ahead. Displayed liquidity at our price is not a promise of a fill, and even a likely fill can be a bad trade.

Example Take the constructed inputs a=4a = 4, μ=3\mu = 3 per minute and h=2h = 2 minutes. The order needs five service events where six are expected, and the previous card gives the complement 115 e−6115\, e^{-6}, so

P(Tfill≤2)=1−115 e−6≈0.71494.\begin{aligned} &\mathbb{P}(T_{\text{fill}} \le 2) \\[0.4em] &\quad = 1 - 115\, e^{-6} \\[0.4em] &\quad \approx 0.71494 . \end{aligned}

With μ=2\mu = 2 per minute the expected count is 44, the complement is e−4(1+4+8+323+323)=1033e−4≈0.62884e^{-4}\bigl(1 + 4 + 8 + \tfrac{32}{3} + \tfrac{32}{3}\bigr) = \tfrac{103}{3} e^{-4} \approx 0.62884, and the fill probability falls to about 0.371160.37116. At μ=3\mu = 3, one more unit ahead lowers it to about 0.554320.55432, and a horizon of 33 minutes raises it to about 0.945040.94504.

Erlang distribution

Definition For an integer n≥1n \ge 1 and a rate μ>0\mu > 0, the Erlang distribution with shape nn and rate μ\mu is the law of the sum of nn independent exponential random variables with rate μ\mu. It is the gamma distribution with integer shape nn, with density

f(t)=μntn−1e−μt(n−1)!,t≥0,f(t) = \frac{\mu^n t^{n-1} e^{-\mu t}}{(n-1)!}, \quad t \ge 0,

mean n/μn/\mu and variance n/μ2n/\mu^2. In a Poisson process with rate μ\mu, the time TnT_n of the nn-th event is the sum of nn interarrival times, so it has this law. The events {Tn≤h}\{T_n \le h\} and {Nh≥n}\{N_h \ge n\} coincide, which gives the distribution function

F(h)=P(Tn≤h)=1−e−μh∑k=0n−1(μh)kk!.\begin{aligned} F(h) &= \mathbb{P}(T_n \le h) \\[0.4em] &= 1 - e^{-\mu h} \sum_{k=0}^{n-1} \frac{(\mu h)^k}{k!} . \end{aligned}

In the pure queue-depletion model the fill time is Tfill=Ta+1T_{\text{fill}} = T_{a+1}, which is Erlang with shape a+1a + 1 and rate μ\mu, so

E[Tfill]=a+1μ.\mathbb{E}[T_{\text{fill}}] = \frac{a + 1}{\mu} .

The count threshold and the sum of waiting times describe the same restricted queue. If a price change can end the queue, if the service rate changes over time or with the state, or if one event can remove several units, the fill time is in general no longer Erlang.

In plain words Each unit ahead takes an exponential time to clear, with mean 1/μ1/\mu, and our own unit takes one more. The average wait is the number of units to be served divided by the service rate, but the wait is random, and against a deadline its spread matters as much as its mean.

Example With a=4a = 4 and μ=3\mu = 3 per minute, the fill time has mean 5/3≈1.6675/3 \approx 1.667 minutes, standard deviation 5/3≈0.745\sqrt{5}/3 \approx 0.745 minutes and median about 1.5571.557 minutes. The mean is below the two-minute horizon, yet the order misses that horizon with probability about 0.285060.28506. Integrating the density with n=5n = 5 from 00 to 22 returns 0.714940.71494, the fill probability of the previous card. With μ=2\mu = 2 the mean fill time is 2.52.5 minutes.

Birth-death process

Definition A birth-death process is a continuous-time Markov chain on the nonnegative integers whose jumps are +1+1, called births, or −1-1, called deaths. In a fill model with births, let ntn_t be the number of units still to be served including ours, so n0=a+1n_0 = a + 1. In every state n≥1n \ge 1, a unit is placed ahead of ours at rate α≥0\alpha \ge 0, which raises nn by one, and a service event at rate μ>0\mu > 0 lowers nn by one. State 00 means that our order has filled, and it is absorbing, because no unit can be placed ahead of an order that has already traded. The probabilities pn(t)=P(nt=n)p_n(t) = \mathbb{P}(n_t = n) solve the Kolmogorov forward equations

pn′=α pn−1+μ pn+1−(α+μ) pn,n≥2,p1′=μ p2−(α+μ) p1,p0′=μ p1,\begin{aligned} p_n' &= \alpha\, p_{n-1} + \mu\, p_{n+1} \\[0.4em] &\quad - (\alpha + \mu)\, p_n, \quad n \ge 2, \\[0.4em] p_1' &= \mu\, p_2 - (\alpha + \mu)\, p_1, \\[0.4em] p_0' &= \mu\, p_1, \end{aligned}

with pa+1(0)=1p_{a+1}(0) = 1 and pn(0)=0p_n(0) = 0 for n≠a+1n \ne a + 1, and the fill probability by time hh is p0(h)p_0(h). Each equation adds the probability flow into state nn, at the rates of the states it comes from, and subtracts the flow out of nn. State 11 receives no births, because state 00 produces none. With rates αn\alpha_n and μn\mu_n that depend on the state, the interior equation becomes

pn′=αn−1 pn−1+μn+1 pn+1−(αn+μn) pn.\begin{aligned} p_n' &= \alpha_{n-1}\, p_{n-1} \\[0.4em] &\quad + \mu_{n+1}\, p_{n+1} \\[0.4em] &\quad - (\alpha_n + \mu_n)\, p_n . \end{aligned}

Setting α=0\alpha = 0 recovers the pure queue-depletion model. Births ahead need a stated mechanism, such as a loss of priority or a rule for hidden liquidity, because under FIFO a new order at our price joins behind us. Trades that remove several units at once, cancellations ahead whose rate depends on how many units are ahead, and price changes that end the queue before a fill call for a different generator and possibly a larger state space, for example a second absorbing state for an order that can no longer fill. None of them is captured by refitting a constant μ\mu.

In plain words Births push the order back in line and services move it forward. The order fills when the number of units still to be served reaches zero, and once it is there nothing can undo the fill.

Example Keep a=4a = 4 and μ=3\mu = 3 per minute and add births ahead at the constructed rate α=1\alpha = 1 per minute. Solving the forward equations numerically from n0=5n_0 = 5 gives p0(2)≈0.4623p_0(2) \approx 0.4623, against 0.714940.71494 without births, and p0(5)≈0.9272p_0(5) \approx 0.9272. With α=0\alpha = 0 the same equations return 0.714940.71494, the Poisson tail of the fill probability card.

Check yourself

  1. A one-unit buy order rests behind a=2a = 2 units, and eligible removals form a Poisson process with rate 1.51.5 per minute. Under the pure queue-depletion assumptions, find the probability that the order fills within one minute and its expected fill time.
Show the answer

The expected count over one minute is μh=1.5\mu h = 1.5, and the order needs at least a+1=3a + 1 = 3 service events. The fill probability is

1−e−1.5(1+1.5+1.125)=1−3.625 e−1.5≈0.19115.\begin{aligned} &1 - e^{-1.5}\bigl(1 + 1.5 + 1.125\bigr) \\[0.4em] &= 1 - 3.625\, e^{-1.5} \\[0.4em] &\approx 0.19115 . \end{aligned}

The fill time is Erlang with shape 33 and rate 1.51.5, so its mean is 3/1.5=23/1.5 = 2 minutes. The order fills within one minute only about 1919 percent of the time.

  1. New buy orders at our price arrive at 33 per minute. A colleague treats them as births ahead of the order of the examples, with α=3\alpha = 3 and μ=3\mu = 3 per minute and a=4a = 4, solves the forward equations and reports a two-minute fill probability of about 0.15050.1505. What is wrong, and what does the pure model give?
Show the answer

Under FIFO priority a new order at our price joins the back of the queue, behind our order, so it does not change the number of units ahead and is not a birth in the fill model. A birth needs a mechanism that really puts a unit ahead of ours, such as a loss of priority or a hidden-liquidity rule. If the other pure-depletion assumptions hold, the fill probability stays

1−115 e−6≈0.71494.1 - 115\, e^{-6} \approx 0.71494 .

The colleague’s chain also has the wrong qualitative behavior. With α=μ\alpha = \mu the order still fills eventually with probability 11, but the expected fill time is infinite, while the actual model has the finite mean 5/35/3 minutes. Counting every book update as a step for or against the order confuses the events that move its position with those that do not.

  1. In the birth-death model with a=4a = 4 and μ=3\mu = 3 per minute, show that α=1\alpha = 1 per minute gives an expected fill time of 2.52.5 minutes. Then take α=4\alpha = 4 per minute and find the probability that the order ever fills.
Show the answer

Let ss be the expected time for the number of units still to be served to fall by one. It is the same in every state n≥1n \ge 1, because the rates do not depend on nn. The first event comes after a mean time 1/(α+μ)1/(\alpha + \mu). With probability μ/(α+μ)\mu/(\alpha + \mu) it is a service and the step is done, and with probability α/(α+μ)\alpha/(\alpha + \mu) it is a birth, after which the count must fall by two, which takes 2s2s on average. Hence

s=1α+μ+αα+μ 2s.s = \frac{1}{\alpha + \mu} + \frac{\alpha}{\alpha + \mu}\, 2s .

When α<μ\alpha < \mu the time ss is finite and the equation gives s=1/(μ−α)=1/2s = 1/(\mu - \alpha) = 1/2 minute. The order needs five such steps, so E[Tfill]=5/(μ−α)=2.5\mathbb{E}[T_{\text{fill}}] = 5/(\mu - \alpha) = 2.5 minutes, against 5/35/3 minutes without births.

For α=4\alpha = 4 let rr be the probability that the count ever falls by one. The same first-step argument gives r=μ/(α+μ)+αr2/(α+μ)r = \mu/(\alpha + \mu) + \alpha r^2/(\alpha + \mu), that is αr2−(α+μ)r+μ=0\alpha r^2 - (\alpha + \mu) r + \mu = 0, with roots 11 and μ/α\mu/\alpha. The hitting probability is the smallest nonnegative root, so r=3/4r = 3/4. By the strong Markov property the five steps down are independent, which gives

(34)5=2431024≈0.2373.\Bigl(\frac{3}{4}\Bigr)^{5} = \frac{243}{1024} \approx 0.2373 .

With probability about 0.76270.7627 the order never fills, so its expected fill time is infinite. Births faster than services make the queue drift away from the fill.

How this lesson connects