Theorem (Delta method): Let \(g: \real^d \to \real^k\) such that \(\nabla g\) is continuous in a neighborhood of \(\bm \in \real^d\) and suppose \(\sqrt{n}(\x_n - \bm) \inD \x\). Then
\[\sqrt{n}(g(\x_n) - g(\bm)) \inD \nabla g(\bm) \Tr \x.\]Proof: We begin by noting that if \(\sqrt{n}(\x_n - \bm) \inD \x\), we must have \(\x_n \inP \bm\); showing this is a good exercise. Now, letting \(N_r(\bm)\) denote the neighborhood of \(\bm\) where \(\nabla g\) is continuous,
\[\begin{alignat*}{2} & \x_n \in |N_r \implies \exists \bar{\x}_n: g(\x_n) = g(\bm) + \nabla g(\bar{\x}_n) \Tr (\x_n - \bm) &\hspace{3em}& \href{taylor-series.html}{\textnormal{Taylor series}} \\ \tag*{$\tcirc{1}$} & \phantom{\x_n \in |N_r \implies} \sqrt{n} \{g(\x_n) - g(\bm)\} = \nabla g(\bar{\x}_n) \Tr [\sqrt{n}(\x_n - \bm)] \\ & \Pr\{\x_n \in |N_r\} \to 1 && \x_n \inP \bm \\ & \bar{\x}_n \inP \bm && \bar{\x}_n \in LS(\bm, \x_n); \x_n \inP \bm \\ & \nabla g(\bar{\x}_n) \inP \nabla g(\bm) && \href{continuous-mapping-theorem.html}{\text{CMT}}; \nabla g \textnormal{ is continuous} \\ & \sqrt{n}(g(\x_n) - g(\bm)) \inD \nabla g(\bm) \Tr \x && \href{slutsky.html}{\text{Slutsky}}; \tcirc{1}; \sqrt{n}(\x_n - \bm) \inD \x \end{alignat*}\]It is worth noting that while applying a Taylor series expansion requires conditions, these conditions only need to be met with probability tending to 1 in order to establish convergence in distribution. This is a consequence of the asymptotic equivalence lemma.
Typically, the distribution involved is the normal distribution as a consequence of the central limit theorem, leading directly to this corollary.
Corollary (Delta method): Let \(g: \real^d \to \real^k\) such that \(\nabla g\) is continuous in a neighborhood of \(\bm \in \real^d\) and suppose \(\sqrt{n}(\x_n - \bm) \inD \Norm(\zero, \bS)\). Then
\[\sqrt{n}(g(\x_n) - g(\bm)) \inD \Norm(\zero, \nabla g(\bm) \Tr \bS \nabla g(\bm)).\]