Convergence of random variables

SOURCE EDITOR ONLY! This page uses LaTeX markup to display mathematical formulas. Editing the page with the VisualEditor or Classic rich-text editor disrupts the layout. Do not even switch to one of these editors while editing the page! For help with mathematical symbols, see Mathematical symbols and expressions.

In probability theory, there exist several different notions of convergence of random variables. The convergence of sequences of random variables to some limit random variable is an important concept in probability theory, and its applications to statistics and stochastic processes. The same concepts are known in more general mathematics as stochastic convergence and they formalize the idea that a sequence of essentially random or unpredictable events can sometimes be expected to settle down into a behavior that is essentially unchanging when items far enough into the sequence are studied. The different possible notions of convergence relate to how such a behavior can be characterized: two readily understood behaviors are that the sequence eventually takes a constant value, and that values in the sequence continue to change but can be described by an unchanging probability distribution.

Background

"Stochastic convergence" formalizes the idea that a sequence of essentially random or unpredictable events can sometimes be expected to settle into a pattern. The pattern may for instance be

• Convergence in the classical sense to a fixed value, perhaps itself coming from a random event
• An increasing similarity of outcomes to what a purely deterministic function would produce
• An increasing preference towards a certain outcome
• An increasing "aversion" against straying far away from a certain outcome
• That the probability distribution describing the next outcome may grow increasingly similar to a certain distribution

Some less obvious, more theoretical patterns could be

• That the series formed by calculating the expected value of the outcome's distance from a particular value may converge to 0
• That the variance of the random variable describing the next event grows smaller and smaller.

These other types of patterns that may arise are reflected in the different types of stochastic convergence that have been studied. The three main types are listed in this article.

Convergence in distribution

With this mode of convergence, we increasingly expect to see the next outcome in a sequence of random experiments becoming better and better modeled by a given probability distribution.

Convergence in distribution is the weakest form of convergence typically discussed, since it is implied by all other types of convergence mentioned in this article. Although convergence in distribution is very frequently used in practice, it only plays a minor role for the purposes of this wiki.

Convergence in probability

The basic idea behind this type of convergence is that the probability of an “unusual” outcome becomes smaller and smaller as the sequence progresses.

The concept of convergence in probability is used very often in statistics.

Definition

A sequence $\{ X_n \}$ of random variables converges in probability towards the random variable $X$ if for all $\varepsilon > 0$

$\displaystyle\lim_{n\to\infty}\mathrm P\big(|X_n-X| > \varepsilon\big) = 0.$

Convergence in probability is denoted by adding the letter $\Rho$ over an arrow indicating convergence, or using the $\operatorname{plim}$ probability limit operator:

$X_n \ \xrightarrow{\mathrm P}\ X,\qquad X_n \ \xrightarrow{p}\ X,\qquad \underset{n\to\infty}{\operatorname{plim}}\, X_n = X$

Properties

• Convergence in probability implies convergence in distribution.
• In the opposite direction, convergence in distribution implies convergence in probability when the limiting random variable $X$ is a constant.
• Convergence in probability does not imply almost sure convergence.
• Convergence in probability is the type of convergence established by the weak law of large numbers.

Almost sure convergence

This is the type of stochastic convergence that is most similar to pointwise convergence known from elementary real analysis.

Definition

To say that the sequence $X_n$ converges almost surely or almost everywhere or with probability 1 or strongly towards $X$ means that

$\mathrm P \left( \displaystyle\lim_{n\to\infty}\! X_n = X \right) = 1$.

This means that the values of $X_n$ approach the value of $X$, in the sense that events for which $X_n$ does not converge to $X$ have probability zero. Using the probability space $(\Omega, \mathcal F, P)$ and the concept of the random variable as a function from $\Omega$ to $\R$, this is equivalent to the statement

$\mathrm P \Big( \omega \in \Omega : \displaystyle\lim_{n \to \infty} X_n(\omega) = X(\omega) \Big) = 1$.

Almost sure convergence is often denoted by adding the letters $\mathrm{a. s.}$ over an arrow indicating convergence:

$\overset{}{X_n \, \xrightarrow{\mathrm{a.\ s.}} \, X}$

Properties

• Almost sure convergence implies convergence in probability, and hence implies convergence in distribution. It is the notion of convergence used in the strong law of large numbers.

See also

• Almost surely