How the derivative of a(θ) is equal to the mean¶
Definition¶
Given that we know:
\[
\begin{align}
\exp(a(\theta)) &= \int_{\cal Y} \exp(\theta y + b(y)) dy \ \ \ \ \ \ \ \ \ \ \ \ (1)\\
a(\theta)) &= \log \int_{\cal Y} \exp(\theta y + b(y)) dy \ \ \ \ \ \ (2)
\end{align}
\]
We take the derivative of \(a(\theta)\) with respect to \(\theta\):
\[
\frac{\partial a(\theta)}{\partial \theta} = \frac{\int_{\cal Y} y \cdot \exp(\theta y+ b(y))dy}{\int_{\cal Y} \exp(\theta y+ b(y))dy}
\]
From equation (1), we substute in \(a(\theta)\):
\[
\frac{\partial a(\theta)}{\partial \theta} = \frac{\int_{\cal Y} y \cdot \exp(\theta y+ b(y))dy}{\exp(a(\theta))}
\]
Here since \(\exp(a(\theta))\) is a constant with respect to \(y\):
\[
\frac{\partial a(\theta)}{\partial \theta} = \frac{1}{\exp(a(\theta))}\int_{\cal Y} y \cdot \exp(\theta y+ b(y))dy
\]
\[
\frac{\partial a(\theta)}{\partial \theta} = \int_{\cal Y} y \cdot \exp(\theta y+ b(y)) \cdot \exp(-a(\theta) dy
\]
\[
\frac{\partial a(\theta)}{\partial \theta} = \int_{\cal Y} y \cdot \exp(\theta y - a(\theta)+ b(y)) dy
\]
But we know that \(p(y| \theta) = \exp(\theta y - a(\theta)+ b(y))\), so:
\[
\frac{\partial a(\theta)}{\partial \theta} = \int_{\cal Y} y \cdot p(y|\theta)dy = \mathbb{E}[Y]
\]