The derivative of the log-likelihood is known as the score:
\[\u(\bt) = \nabla \ell(\bt|\x).\]The score, along with the information, is a critical component in obtaining quadratic approximations to log-likelihood functions.
Note that
- The score \(\u\) is a function of \(\theta\).
- For any given \(\bt\), \(\u(\bt)\) is a random variable, as it depends on the data \(\x\); usually suppressed in notation.
- For independent observations, the score of the entire sample is the sum of the scores for the individual observations:
-
By the chain rule,
\[\u(\bt) = \frac{\nabla_{\bt} p(x|\bt)}{p(x|\bt)},\]where \(p\) is the density function.
Score equations
If the likelihood is regular, we can find \(\bth\) by setting the gradient equal to zero; the MLE is the solution to the equation(s)
\[\u(\bt) = \zero;\]this system of equations is known as the score equation(s) or sometimes the likelihood equation(s).