In many optimization problems arising from scientific, engineering and artificial intelligence
applications, objective and constraint functions are available only as the output of a black …

Modern problems in AI or in numerical analysis require nonsmooth approaches with a
flexible calculus. We introduce generalized derivatives called conservative fields for which …

Automatic differentiation, as implemented today, does not have a simple mathematical
model adapted to the needs of modern machine learning. In this work we articulate the …

Differentiation lies at the core of many machine-learning algorithms, and is well-supported
by popular autodiff systems, such as TensorFlow and PyTorch. Originally, these systems …

A new method for evaluating generalized derivatives in nonsmooth problems is reviewed.
Lexicographic directional (LD-) derivatives are a recently developed tool in nonsmooth …

Any piecewise smooth function that is specified by an evaluation procedure involving
smooth elemental functions and piecewise linear functions like and can be represented in …

In the paper [Optim. Methods Softw., 31 (2016), pp. 904--930] we derived first order (KKT)
and second order (second order sufficiency condition (SOSC)) optimality conditions for …

A recent nonsmooth vector forward mode of algorithmic differentiation (AD) computes
Nesterov's L-derivatives for nonsmooth composite functions; these L-derivatives provide …

We present a new algorithm, called manifold sampling, for the unconstrained minimization of
a nonsmooth composite function h∘F when h has known structure. In particular, by …

We present an optimization method for Lipschitz continuous, piecewise smooth (PS)
objective functions based on successive piecewise linearization. Since, in many realistic …