Theorem (Law of large numbers): Let \(\x, \x_1, \x_2, \ldots\) be independently and identically distributed random vectors such that \(\Ex\norm{\x}^2 < \infty\). Then \(\bar{\x}_n \inQM \bm\), where \(\bm = \Ex(\x)\).
Proof:
\(\begin{alignat*}{2} \Ex\norm{\bar{\x} - \bm}_2^2 &= \Ex\{ (\bar{\x} - \bm) \Tr (\bar{\x} - \bm)\} \\ &= n^{-2} \Ex\left\{ \sum_{i,j} (\x_i - \bm) \Tr (\x_j - \bm)\right\} &\hspace{8em}& \bar{\x} - \bm = \tfrac{1}{n}\sum_i (\x_i-\bm) \\ &= n^{-2} \sum_{i,j} \Ex\left\{ (\x_i - \bm) \Tr (\x_j - \bm)\right\} && \textnormal{Linearity of expectation} \\ &= n^{-2} \sum_{i} \Ex\left\{ (\x_i - \bm) \Tr (\x_i - \bm)\right\} && \Ex\left\{ (\x_i - \bm) \Tr (\x_j - \bm)\right\} = \trace\{\textnormal{Cov}(\x_i, \x_j)\}, \\ & && \textnormal{which is 0 if } \x_i \ind \x_j \\ &= n^{-1} \Ex\left\{ (\x - \bm) \Tr (\x - \bm)\right\} && \Ex \x_i, \Var \x_i \textnormal{ same for all } i \\ &= n^{-1} \Ex\left\{ \x \Tr \x - 2 \bm \Tr \x + \bm \Tr \bm \right\} \\ &\to 0 && \Ex\norm{\x}^2 \le \infty \end{alignat*}\)
Note that this proof, unlike the usual law of large numbers, does not actually require \(\{\x_i\}\) to be independent or identically distributed, only that the terms in the sum are uncorrelated and have the same mean and variance.