Proximal Update¶
Definition¶
The proximal gradient update is in two steps, which are the same steps in Proximal Methods:
\[
\begin{align} \\
&1. \ \tilde{\mathbf{w}}_{t+1} = \mathbf{w}_{t} + \eta \mathbf{g}_{t} \\
&2. \ \mathbf{w}_{t+1} = \arg\!\min_{\mathbf{w} \in \mathbb{R}^d} \frac{1}{2}||\mathbf{w} - \tilde{\mathbf{w}}_{t+1}||_{2}^2 + \eta_{t}r(\mathbf{w})
\end{align}
\]
The second equation is called the Proximal Operator