Almost sure convergence

Definition: A sequence of random variables $x_{n}$ converges almost surely to the random variable $x$ , denoted $x_{n} \overset{as}{⟶} x$ , if

P {lim_{n \to \infty} x_{n} = x} = 1.

An equivalent definition is given below.

Definition: A sequence of random variables $x_{n}$ converges almost surely to the random variable $x$ , denoted $x_{n} \overset{as}{⟶} x$ , if for every $ϵ > 0$ ,

P {‖ x_{k} - x ‖ < ϵ for all k \geq n} \to 1

as $n \to \infty$

Comparing this definition to that of convergence in probability, it is clear that almost sure convergence implies convergence in probability.

Theorem: $x_{n} \overset{as}{⟶} x ⟹ x_{n} \overset{p}{⟶} x$ .

If $\hat{θ} \overset{as}{⟶} θ$ , then $\hat{θ}$ is said to be a strongly consistent estimator of $θ$ . The sample mean, for example, is a strongly consistent estimator of $E (X)$ ; this result is known as the strong law of large numbers.

Theorem (Strong law of large numbers): Let $x, x_{1}, x_{2}, \dots$ be independently and identically distributed random vectors such that $E ‖ x ‖ < \infty$ . Then ${\bar{x}}_{n} \overset{as}{⟶} μ$ , where $μ = E (x)$ .