Package org.nd4j.linalg.learning.regularization

Interface Summary

Interface	Description
Regularization	Regularization API for magnitude-based regularization techniques such as: L1Regularization L2Regularization WeightDecay Implementations should have the following features: 1.

Class Summary

Class	Description
L1Regularization	L1 regularization: Implements updating as follows: L = loss + l1 * sum_i abs(w[i]) {@code w[i] -= updater(gradient[i] + l1 * sign(w[i])) - where sign(w[i]) is +/- 1 Note that L1 regularization is applied before the updater (Adam/Nesterov/etc) is applied.
L2Regularization	L2 regularization: very similar to WeightDecay, but is applied before the updater is applied, not after.
WeightDecay	WeightDecay regularization: Updater is not applied to the regularization term gradients, and (optionally) applies the learning rate.

Enum Summary

Enum	Description
Regularization.ApplyStep	ApplyStep determines how the regularization interacts with the optimization process - i.e., when it is applied relative to updaters like Adam, Nesterov momentum, SGD, etc.