Hitting Times and Absorption

Definition Let $X_0, X_1, X_2, \dots$ be a time-homogeneous Markov chain on a state space $S$ with transition matrix $P$. A state $i$ is absorbing if $P_{ii} = 1$, so that $P_{ij} = 0$ for every $j \ne i$. Once the chain is in an absorbing state, it stays there at every later step. In a chain with absorbing states, entering one of them is called absorption. Write $P_i$ and $E_i$ for probabilities and expectations when the chain starts in state $i$, that is, with $X_0 = i$.

Definition For a nonempty set $D$ of target states, the hitting time of $D$ is

the first time the chain is in $D$, with $T_D = \infty$ when the chain never enters $D$. Time $0$ counts, so $T_D = 0$ when $X_0 \in D$. For a single state B write $T_B$. The variable $\min\{ n \ge 1 : X_n \in D \}$, which ignores time $0$, is a different random time. It agrees with $T_D$ whenever the chain starts outside $D$.

Definition For a target set $D$ and a start $i$, the hitting probability is

the probability that the chain started at $i$ ever enters $D$. It is a different quantity from the probability $P_i(T_D \le n)$ of reaching $D$ by a deadline $n$ and from the probability $P_i(X_n \in D)$ of being in $D$ at step $n$. The events $\{T_D \le n\}$ grow with $n$ and their union is $\{T_D < \infty\}$, so $P_i(T_D \le n)$ increases to $h_i$ as $n \to \infty$. When every state of $D$ is absorbing, being in $D$ at step $n$ is the same event as $\{T_D \le n\}$.

Definition First-step analysis finds hitting quantities by conditioning on the first step $X_1$. For a target set $D$, the hitting probabilities $h_i = P_i(T_D < \infty)$ satisfy, for every state $i$ outside $D$,

The reason is that the first step goes to $j$ with probability $P_{ij}$, and by the Markov property and time homogeneity the chain then behaves like a new chain started at $j$. Values that follow directly from the meaning of the question, rather than from this equation, are boundary conditions. For hitting probabilities they are $h_i = 1$ for every $i$ in $D$, and $h_i = 0$ for every state $i$ from which $D$ cannot be reached. On a finite state space, the equations together with both kinds of boundary condition have exactly one solution.

Definition For a target set $D$, let $k_i = E_i[T_D]$. When $D$ is the set of absorbing states, $k_i$ is the expected time to absorption from $i$. The boundary conditions are $k_i = 0$ for $i$ in $D$, not $1$ as for probabilities, and for $i$ outside $D$ first-step analysis gives

where the $1$ counts the step just taken and the sum is the expected remaining time. These equations should be treated as ordinary linear equations only after checking that the $k_i$ involved are finite. One check is the tail formula $E[T] = \sum_{n=0}^{\infty} P(T > n)$, valid for every random variable $T$ whose values are nonnegative integers or $\infty$. On a finite state space, if $D$ can be reached from every state, then every $k_i$ is finite.

Definition A hitting time $T_D$ equals $\infty$ on every path that never enters $D$. If $P_i(T_D = \infty) > 0$, then $E_i[T_D] = \infty$, because $E_i[T_D] \ge n \, P_i(T_D = \infty)$ for every $n$. When $h_i > 0$, the conditional mean $E_i[T_D \mid T_D < \infty]$ averages only over the paths that reach $D$, and it can be finite even when $E_i[T_D]$ is infinite. The mean can also be infinite when $P_i(T_D < \infty) = 1$. On a finite state space this cannot happen, but on the integers it does, for the symmetric simple random walk started at $1$ with target $0$.