More details on log-normalizer function a(θ)¶
Definition¶
\(a(\theta)\) is called the log-normalizer function from this derivation:
Starting from the fact that we must have a probability distribution equal to 1:
\[
\int_{\cal{Y}} p(y|\theta)dy \ = \int_{\cal{Y}} \exp(\theta y - a(\theta) + b(y))dy = 1
\]
Then we factor out an \(\exp(-a(\theta))\):
\[
\exp(-a(\theta)) \cdot \int_{\cal{Y}} \exp(\theta y-b(y))dy = 1
\]
Move \(\exp(-a(\theta))\) to the other side:
\[
\exp(a(\theta)) = \int_{\cal{Y}} \exp(\theta y-b(y))dy
\]
Get rid of exp():
\[
a(\theta) = \log \int_{\cal{Y}} \exp(\theta y-b(y))dy
\]
This is basically saying \(a(\theta)\) ensures we have a valid distribution.