Poisson normalizing constant

This is an example using the Poisson distribution illustrate exponential tilting.

Let $p_{0} (x)$ denote the “standard” Poisson distribution ( $μ = 1$ ):

p_{0} (x) = e^{- 1} / x!

and $\tilde{p} (x | θ)$ its exponentially tilted form:

\tilde{p} (x | θ) = p_{0} (x) e^{θ x} .

This isn’t a proper distribution, hence the notation $\tilde{p} (x | θ)$ , but it would be if we determined the normalizing constant, which I will denote $\exp {ψ (θ)}$ :

\begin{aligned} \exp {ψ (θ)} & = \sum_{x = 0}^{\infty} e^{- 1} e^{θ x} / x! \\ = e^{- 1} \exp {e^{θ}} & e^{x} = \sum_{n = 0}^{\infty} x^{n} / n! \\ = \exp {e^{θ} - 1} \end{aligned}

Thus $ψ (θ) = e^{θ} - 1$ and

\begin{aligned} p (x | θ) & = p_{0} (x) e^{θ x - ψ (θ)} \\ = \exp {- 1 + θ x - e^{θ} + 1} / x! \\ = \exp {θ x - e^{θ}} / x!, \end{aligned}

or, letting $μ = e^{θ}$ ,

p (x | θ) = e^{- μ} μ^{x} / x!,

the familiar form of the Poisson distribution. Note the main idea, that we started with a base distribution and ended up with a family of derived distributions, arising from the process of exponential tilting.