This comprehensive survey explored the evolving landscape of generative Artificial
Intelligence (AI), with a specific focus on the transformative impacts of Mixture of Experts …

Optimization in multi-task learning (MTL) is more challenging than single-task learning
(STL), as the gradient from different tasks can be contradictory. When tasks are related, it …

Abstract Sparsely activated Mixture-of-Experts (MoE) is becoming a promising paradigm for
multi-task learning (MTL). Instead of compressing multiple tasks' knowledge into a single …

Learning discriminative task-specific features simultaneously for multiple distinct tasks is a
fundamental problem in multi-task learning. Recent state-of-the-art models consider directly …

Scaling model parameters improves model quality at the price of high computation
overhead. Sparsely activated models, usually in the form of Mixture of Experts (MoE) …

Understanding the parameter estimation of softmax gating Gaussian mixture of experts has
remained a long-standing open problem in the literature. It is mainly due to three …

Reconstructing the viewed images from human brain activity bridges human and computer
vision through the Brain-Computer Interface. The inherent variability in brain function …

The Mixture-of-Experts (MoE) approach has demonstrated outstanding scalability in multi-
task learning including low-level upstream tasks such as concurrent removal of multiple …

Previous multi-task dense prediction methods based on the Mixture of Experts (MoE) have
received great performance but they neglect the importance of explicitly modeling the global …

In this work we present Omni-SMoLA a multimodal architecture that mixes many multi-modal
experts efficiently and achieves both high specialist and generalist performance. In contrast …