Generalized Linear Models (GLM)¶
Definition¶
Read GLM Preface to better understand.
GLM's allow us to extend linear regression prediction models past Gaussian distributions and have nonlinear dependencies between features and targets.
Let's consider the natural exponential family, which is a class of probability distributions:
\(\theta \in \mathbb{R}\) is the natural parameter to the distribution. And \(\theta = \mathbf{xw}\). \(a(\theta): \mathbb{R} \to \mathbb{R}\) is the log-normalizer function. \(b(y)\) is irrelevant.
GLM functions are also convex.
More details on log-normalizer function a(θ)
How the derivative of a(θ) is equal to the mean
To formalize GLM's:
\(g\) is the Transfer Function (Machine Learning), which is also the derivative of \(a(\theta)\). For Poisson, \(g = \exp(\mathbf{xw})\) For Logistic Regression, \(g\) is the sigmoid function
In order for a GLM to be valid, its transfer function must be invertible.