Theorem: Suppose \(X_1, \ldots, X_n\) are iid with density \(p(x|\bts)\) and that derivatives up to second order can be passed under the integral sign in \(\int dP(x|\bt)\). Then
\[\tfrac{1}{\sqrt{n}}\u(\bt^*) \inD \Norm(\zero, \fI(\bt^*))\]Proof:
\[\begin{alignat*}{2} &\Var \u(\bts) = \fI(\bts) &\hspace{4em}& \href{fisher-information.html}{\text{Def} \fI} \\ &\sqrt{n} \{\bar{\u}(\bt^*) - \Ex\u(\bt^*)\} \inD \Norm(\zero, \fI(\bt^*)) && \href{central-limit-theorem.html}{\text{CLT}} \text{ (iid)} \\ & \tfrac{1}{\sqrt{n}}\u(\bt^*) \inD \Norm(\zero, \fI(\bt^*)) && \href{score-expectation.html}{\Ex \u(\bts) = \zero}; \bar{\u} = \u/n \\ \end{alignat*}\]Corollaries
The above result can be stated in a variety of ways.
Corollary (#1): Under the same conditions as the original theorem, if \(\fI(\bts)\) is positive definite, then
\[\tfrac{1}{\sqrt{n}} \fI^{-1/2}(\bts)\u(\bts) \inD \Norm(\zero, \I).\]Proof: Continuous mapping theorem.
Corollary (#2): Under the same conditions as the first corollary,
\[\oI_n(\bt^*)^{-1/2}\u(\bt^*) \inD \Norm(\zero, \I).\]Proof:
\[\begin{alignat*}{2} \tag*{$\tcirc{1}$} \tfrac{1}{n}\oI_n(\bts) \inP &\fI(\bts) &\hspace{4em}& \href{weak-law-of-large-numbers.html}{\text{LLN}} \text{ (iid)} \\ \oI_n(\bt^*)^{-1/2}\u(\bt^*) &= [n \tfrac{1}{n} \oI_n(\bts)]^{-1/2} \u(\bts) \\ &= \underbrace{[\tfrac{1}{n} \oI_n(\bts)]^{-1/2} \fI(\bts)^{1/2}}_{\A} \underbrace{\tfrac{1}{\sqrt{n}} \fI(\bts)^{-1/2} \u(\bts)}_{\b} \\ \end{alignat*}\]At this point, note that \(\A \inP \I\) by \(\tcirc{1}\) and CMT, while \(\b \inD \Norm(\zero, \I)\) by our first corollary. Thus, the result
\[\oI_n(\bt^*)^{-1/2}\u(\bt^*) \inD \Norm(\zero, \I).\]holds by Slutsky’s theorem.
Finally, note that Corollary #2 holds for any consistent estimator of \(\fI(\bts)\). For example, if $\tfrac{1}{n}\oI_n(\bth) \inP \fI(\bts)$, we would also have
\[\oI_n(\bth)^{-1/2}\u(\bt^*) \inD \Norm(\zero, \I)\]by the same proof.