Convergence of random variables

In probability theory, there exist several different notions of convergence of random variables. The convergence (in one of the senses presented below) of sequences of random variables to some limiting random variable is an important concept in probability theory, and its applications to statistics and stochastic processes. For example, if the average of ''n'' independent, identically distributed random variables ''Y''_''i'', ''i'' = 1, ..., ''n'', is given by :$X\_n\; =\; \backslash frac\{1\}\{n\}\backslash sum\_\{i=1\}^n\; Y\_i\backslash ,,$ then as ''n'' goes to infinity, ''X''_''n'' converges ''in probability'' (see below) to the common mean, μ, of the random variables ''Y''_''i''. This result is known as the weak law of large numbers. Other forms of convergence are important in other useful theorems, including the central limit theorem. Throughout the following, we assume that (''X''_''n'') is a sequence of random variables, and ''X'' is a random variable, and all of them are defined on the same probability space (Ω, ''F'', P). == Convergence in distribution == Suppose that ''F''₁, ''F''₂, ... is a sequence of cumulative distribution functions corresponding to random variables ''X''₁, ''X''₂, ..., and that ''F'' is a distribution function corresponding to a random variable ''X''. We say that the sequence ''X''_''n'' converges towards ''X'' in distribution, if ::$\backslash lim\_\{n\backslash rightarrow\backslash infty\}\; F\_n(a)\; =\; F(a),$ for every real number ''a'' at which ''F'' is continuity. Since ''F''(a) = Pr(''X'' ≤ a), this means that the probability that the value of ''X'' is in a given range is very similar to the probability that the value of ''X''_''n'' is in that range, provided ''n'' is large enough. Convergence in distribution is often denoted by adding the letter 'D' over an arrow indicating convergence: ::$X\_n\; \backslash ,\; \backslash begin\{matrix\}\; \{\backslash ,\}\_D\; \backslash \backslash \; \{\backslash ,\}^\{\backslash rightarrow\}\; \backslash end\{matrix\}\; \backslash ,\; X.$ Convergence in distribution is the weakest form of convergence, and is sometimes called weak convergence. It does not, in general, imply any other mode of convergence. However, convergence in distribution ''is'' implied by all other modes of convergence mentioned in this article, and hence, it is the most common and often the most useful form of convergence of random variables. It is the notion of convergence used in the central limit theorem and the (weak) law of large numbers. A useful result, which may be employed in conjunction with law of large numbers and the central limit theorem, is that if a function  ''g'': R → R  is continuous, then if  ''X''_''n''  converges in distribution to  ''X'', then so too does  ''g''(''X''_''n'')  converge in distribution to  ''g''(''X''). (This may be proved using Skorokhod's representation theorem.) Convergence in distribution is also called convergence in law, since the word "law" is sometimes used as a synonym of "probability distribution." == Convergence in probability == We say that the sequence ''X''_''n'' converges towards ''X'' in probability if :$\backslash lim\_\{n\backslash rightarrow\backslash infty\}P\backslash left(\backslash left|X\_n-X\backslash right|\backslash geq\backslash varepsilon\backslash right)=0$ for every ε > 0. Convergence in probability is, indeed, the (pointwise convergence) convergence ''of'' probabilities. Pick any ε > 0 and any δ > 0. Let ''P''_''n'' be the probability that ''X''_''n'' is outside a tolerance ε of ''X''. Then, if ''X''_''n'' converges in probability to ''X'' then there exists a value ''N'' such that, for all ''n'' ≥ ''N'', ''P''_''n'' is itself less than δ. Convergence in probability is the notion of convergence used in the weak law of large numbers. Convergence in probability implies convergence in distribution. To prove it, it's convenient to prove the following, simple lemma: === Lemma === Be X, Y random variables, c a real number and ε > 0; then :$P(Y\backslash leq\; c)\backslash leq\; P(X\backslash leq\; c+\backslash varepsilon)+P(\backslash left|Y\; -\; X\backslash right|>\backslash varepsilon)$ In fact, :$P(Y\backslash leq\; c)=P(Y\backslash leq\; c,X\backslash leq\; c+\backslash varepsilon)+P(Y\backslash leq\; c,X>c+\backslash varepsilon)$ :
:$P(X\backslash leq\; a-\backslash varepsilon)\backslash leq\; P(X\_n\; \backslash leq\; a)+P(\backslash left|X\_n\; -\; X\backslash right|>\backslash varepsilon)$
So, we have: :$P(X\backslash leq\; a-\backslash varepsilon)-P(\backslash left|X\_n\; -\; X\backslash right|>\backslash varepsilon)\backslash leq\; P(X\_n\; \backslash leq\; a)\backslash leq\; P(X\backslash leq\; a+\backslash varepsilon)+P(\backslash left|X\_n\; -\; X\backslash right|>\backslash varepsilon)$
Taking the limit for $n\backslash rightarrow\backslash infty$, we obtain:
:$P(X\backslash leq\; a-\backslash varepsilon)\backslash leq\; \backslash lim\_\{n\backslash rightarrow\backslash infty\}\; P(X\_n\; \backslash leq\; a)\backslash leq\; P(X\backslash leq\; a+\backslash varepsilon)$
But $P(X\backslash leq\; a)$ is the cumulative distribution function $F\_X(a)$, which is continuous by hypothesis, that is:
:$\backslash lim\_\{\backslash varepsilon\; \backslash rightarrow\; 0^+\}\; F\_X(a-\backslash varepsilon)=\backslash lim\_\{\backslash varepsilon\; \backslash rightarrow\; 0^+\}\; F\_X(a+\backslash varepsilon)=F\_X(a)$
and so, taking the limit for $\backslash varepsilon\; \backslash rightarrow\; 0^+$, we obtain:
:$\backslash lim\_\{n\backslash rightarrow\backslash infty\}\; P(X\_n\; \backslash leq\; a)=P(X\; \backslash leq\; a)$

== Almost sure convergence == We say that the sequence ''X''_''n'' converges almost surely or almost everywhere or with probability 1 or strongly towards ''X'' if :$P\backslash left(\backslash lim\_\{n\backslash rightarrow\backslash infty\}X\_n=X\backslash right)=1.$ This means that you are virtually guaranteed that the values of ''X''_''n'' approach the value of ''X'', in the sense (see almost surely) that events for which ''X''_''n'' does not converge to ''X'' have probability 0. Using the probability space (Ω, ''F'', P) and the concept of the random variable as a function from Ω to R, this is equivalent to the statement :$P\backslash left(\backslash \{\backslash omega\; \backslash in\; \backslash Omega:\; X\_n(\backslash omega)\; \backslash rightarrow\; X\; \backslash ,\backslash ,\; \backslash mathrm\{as\}\; \backslash ,\backslash ,\; n\; \backslash rightarrow\; \backslash infty\backslash \}\backslash right)\; =\; 1.$ Almost sure convergence implies convergence in probability, and hence implies convergence in distribution. It is the notion of convergence used in the strong law of large numbers. == Convergence in ''r''th mean == We say that the sequence ''X''_''n'' converges '''in ''r''th mean or in the Lp space''' towards ''X'', if ''r'' ≥ 1, E|''X''_''n''| < ∞ for all ''n'', and :$\backslash lim\_\{n\backslash rightarrow\backslash infty\}\backslash mathrm\{E\}\backslash left(\backslash left|X\_n-X\backslash right|^r\backslash right)=0$ where the operator E denotes the expected value. Convergence in ''r''th mean tells us that the expectation of the ''r''th power of the difference between ''X''_''n'' and ''X'' converges to zero. The most important cases of convergence in ''r''th mean are: * When ''X''_''n'' converges in ''r''th mean to ''X'' for ''r'' = 1, we say that ''X''_''n'' converges in mean to ''X''. * When ''X''_''n'' converges in ''r''th mean to ''X'' for ''r'' = 2, we say that ''X''_''n'' converges in mean square to ''X''. Convergence in ''r''th mean, for ''r'' ≥ 1, implies convergence in probability (by Chebyshev's inequality), while if ''r'' > ''s'' ≥ 1, convergence in ''r''th mean implies convergence in ''s''th mean. Hence, convergence in mean square implies convergence in mean. == Converse implications == The chain of implications between the various notions of convergence, above, are noted in their respective sections, but it is sometimes important to establish converses to these implications. No other implications other than those noted above hold in general, but a number of special cases do permit converses: * If ''X''_''n'' converges in distribution to a constant ''c'', then ''X''_''n'' converges in probability to ''c''. * If ''X''_''n'' converges in probability ''X'', and if Pr(|''X''_''n''| ≤ ''b'') = 1 for all ''n'' and some ''b'', then ''X''_''n'' converges in ''r''th mean to ''X'' for all ''r'' ≥ 1. In other words, if ''X''_''n'' converges in probability to ''X'' and all random variables ''X''_''n'' are almost surely bounded above and below, then ''X''_''n'' converges to ''X'' also in any ''r''th mean. * If for all ε > 0, :: :then ''X''_''n'' converges almost surely to ''X''. In other words, if ''X''_''n'' converges in probability to ''X'' sufficiently quickly (''i''.''e''. the above sum converges for all ε > 0), then ''X''_''n'' also converges almost surely to ''X''. == References == # G.R. Grimmett and D.R. Stirzaker (1992). ''Probability and Random Processes, 2nd Edition''. Clarendon Press, Oxford, pp 271--285. ISBN 0198536658. probability theory

Convergence of random variables

==plim== Has anyone else seen the "convergence in probability" function being denoted as plim? I recently saw an old textbook as part of a CFA primer course that used this notation, but I have not had success googling for this notation anywhere. Can someone confirm that this is valid notation? If so, can we add it to this article? - User:DropDeadGorgias User_talk:DropDeadGorgias 17:48, Jun 22, 2004 (UTC) ---I can confirm this notation. My econometrics text (Kmenta, Jan. ''Elements of Econometrics: 2nd Ed.'' University of Michigan Press: Ann Arbor, 1987.) includes this notation.

---

See other meanings of words starting from letter:

C

CA | CB | CD | CE | CF | CG | CH | CI | CJ | CK | CL | CM | CN | CO | CP | CR | CS | CT | CU | CW | CX | CY | CZ |

Words begining with Convergence_of_random_variables:

Convergence_of_random_variables
Convergence_of_random_variables

These materials are based on Wikipedia and licensed under the GNU FDL