Abstract Sparsely-gated Mixture of Expert (MoE), an emerging deep model architecture, has
demonstrated a great promise to enable high-accuracy and ultra-efficient model inference …

Massive data is often considered essential for deep learning applications, but it also incurs
significant computational and infrastructural costs. Therefore, dataset pruning (DP) has …

This paper presents a new approach and algorithm for solving a class of constrained Bi-
Level Optimization (BLO) problems in which the lower-level problem involves constraints …

Large Language Models (LLMs) have highlighted the necessity of effective unlearning
mechanisms to comply with data regulations and ethical AI practices. LLM unlearning aims …

Bilevel optimization has been recently applied to many machine learning tasks. However,
their applications have been restricted to the supervised learning setting, where static …

This work focuses on addressing two major challenges in the context of large-scale
nonconvex Bi-Level Optimization (BLO) problems, which are increasingly applied in …

The trustworthy machine learning (ML) community is increasingly recognizing the crucial
need for models capable of selectively'unlearning'data points after training. This leads to the …

In Federated Learning (FL), the distributed nature and heterogeneity of client data present
both opportunities and challenges. While collaboration among clients can significantly …

Stochastic bilevel optimization (SBO) is becoming increasingly essential in machine
learning due to its versatility in handling nested structures. To address large-scale SBO …

A novel multiscale consensus-based optimization (CBO) algorithm for solving bi-and tri-level
optimization problems is introduced. Existing CBO techniques are generalized by the …