Theorem:
For any probability model
where
Proof:
Furthermore, this quantity is strictly negative by Gibbs’ inequality if identifiability holds.
Theorem:
For any probability model
where
Proof:
Furthermore, this quantity is strictly negative by Gibbs’ inequality if identifiability holds.