Recurrent Stochastic Process

From Qwiki

Qualitatively, a stochastic process on a Banach space is said to be recurrent if it re-visits an arbitrary point in its image an infinite number of times. A process that is not recurrent is said to be transient. To put this into more quantitative terms, we will first state an equivalent definition and then, for inquiring minds, we will prove this equivalence.

1 Quantitative Definition
2 Equivalence to Intuition
3 Example
4 Remarks

Quantitative Definition

Let

ξ(t)

be a stochastic process.

ξ(t)

is recurrent if, for any

ε > 0

and any point

a

,
$\infty = \int_0^\infty dt P(\|\xi(t) - a\| < \epsilon | \xi(0) = a).$

Computationally, this definition is somewhat tractable, but a shortcoming is that it is not clear that it necessarily captures the essence of our intuitive qualitative definition. In the next section, we will show that this definition is indeed equivalent to our more intuitive notions.

Equivalence to Intuition

It should be pretty obvious that if $ξ(t)$ does re-visit a point arbitrarily many times, the equation in the previous section is implied. What is not obvious, however, is that if $ξ(t)$ does not re-visit a point arbitrarily many times, that the equation in the previous section fails to hold. Obvious or not, we will show both cases here.

If we assume that we work in a limit in which $ε$ is very small, we can make the approximation that the time that $ξ(t)$ spends within $ε$ of $a$ each time it visits $a$ is negligible. While not a rigorous assumption, this leads to a nice, clean simplification of the problem. Essentially this amounts to first showing the equivalence for stochastic processes on discrete spaces, and then taking the limit of the spacing between points to zero. If you don't like this, contribute to this Qwiki entry!

Let's define the first-passage density $f a (t)$ that $ξ(t)$ , exiting the norm ball $B ε (a)$ of radius $ε$ about $a$ at time $0$ , re-enters $B ε (a)$ at time $t$ , but not any time sooner. $f a (t)$ is very much like the probability density function of this first-arrival event, but as will soon show, it is not necessarily normalized so we hesitate to call it a probability density function.

We may now see that in our small- $ε$ approximation, the probability that $\xi(t) \in B_\epsilon(a)$ at time $t$ can be expressed as
$P(\|\xi(t) - a\| < \epsilon | \xi(0) = a) =$

$f_a(t) + \int_0^t d\tau f_a(\tau) f_a(t - \tau) + \int_0^t d \tau_1 \int_0^{t-\tau_1} d\tau_2 f_a(\tau_1) f_a(\tau_2) f_a(t - \tau_1 - \tau_2) + \cdots$

which is a sum of multiconvolutions:

$P(\|\xi(t) - a\| < \epsilon | \xi(0) = a) = \sum_{i = 1}^\infty (* f_a)^i(t)$
where the unusual notation

( * f a) i

indicates

f a

convolved with itself

i

times. Why can we write this? It's pretty simple actually --- the probability that $\xi(t)\in B_\epsilon(a)$ is just the probability that

ξ(t)

has first entered

B ε (a)

at time

t

, plus the probability that it entered once some at time

τ < t

and then again in

t - τ

, and so on. We will now use this expression to show the equivalence between our qualitative and quantitative statements of the definition of recurrence.

First let's assume that $ξ(t)$ is recurrent in the qualitative sense. Then it will re-enter $B ε (a)$ in finite time with probability 1, so that $\int_0^\infty dt f_a(t) = 1$ . This makes $f a (t)$ a formal probability density, and consequently it makes $( * f a (t)) i (t)$ a formal probability density as well, representing the probability density for the time required for the $i t h$ entry of $ξ(t)$ into $B ε (a)$ . But then $\int_0^t dt(*f_a)^i(t) = 1$ , so that

$\int_0^\infty dt P(\|\xi(t) - a\| < \epsilon | \xi(0) = \sum_{i=1}^\infty \int_0^\infty dt(*f_a)^i(t) = \infty.$
Half of our work is done, as we have just shown that our qualitative recurrence definition implies our quantitative one.

Now let's assume that $ξ(t)$ is transient in the qualitative sense. Then we necessarily have $\int_0^\infty dt f_a(t) \equiv \beta < 1$ . As before, the quantity $\int_0^\infty dt (*f_a)^i(t)$ is the probability that $ξ(t)$ returns to $B ε (a)$ $i$ times. But this is just $β i$ , so that

$\int_0^\infty dt P(\|\xi(t) - a\| < \epsilon | \xi(0) = \sum_{i=1}^\infty \beta^i = \frac \beta { 1 - \beta } < \infty.$

Note that any norm ball about $a$ can be constructed as a finite union of smaller norm balls. Thus the probability $p(\|\xi(t)-a\| < \epsilon | \xi(0) = a)$ is finite for all $ε$ whenever it is finite for any $ε$ . This completes the proof that our quantitative definition of recurrence is equivalent to our qualitative one.

Example

Recurrence of Brownian Motion

Remarks

The notion of recurrence in random processes most frequently shows up in discussions of states of a Markov chain; a particular state must be either recurrent or transient. This notion can of course be extended to a Markov process on a continuous space. Our definition of a recurrent process is a Markov process where a neighborhood about every point is recurrent.

Retrieved from "http://qwiki.stanford.edu/wiki/Recurrent_Stochastic_Process"

Categories: Concept | Probability