查看文章

The notion of regularization has been widely used as a tool to address a number of problems that are usually encountered in Machine Learning. Improving the performance of an estimator by shrinking the norm of the Minimum Variance Unbiased (MVU) estimator, guarding against overfitting, coping with ill-conditioning, providing a solution to an underdetermined set of equations are some notable examples where regularization has provided successful answers. A notable example is the ridge regression concept, where the LS loss function is combined, in a tradeoff rationale, with the Euclidean norm of the desired solution.

In this chapter, our interest will be on alternatives to the Euclidean norms and in particular the focus will revolve around the ℓ 1 norm; this is the sum of the absolute values of the components comprising a vector. Although seeking a solution to a problem via the ℓ 1 norm regularization of a loss …