ReLU

Definition

For some scalar \(x\), the ReLU function returns \(\max(0,x)\).

ReLU is used in neural networks because it is very simple and easily differentiable.

Example