In recent years, machine learning, especially various deep neural networks, as an emerging
technique for data analysis and processing, has brought novel insights into the development …

In recent years, generative transformers have become increasingly prevalent in the field of
artificial intelligence, especially within the scope of natural language processing. This paper …

The scope of data-driven fault diagnosis models is greatly extended through deep learning
(DL). However, the classical convolution and recurrent structure have their defects in …

Given the massive cost of language model pre-training, a non-trivial improvement of the
optimization algorithm would lead to a material reduction on the time and cost of training …

Deep neural networks often suffer from poor performance or even training failure due to the
ill-conditioned problem, the vanishing/exploding gradient problem, and the saddle point …

Efficiently implementing remote sensing image classification with high spatial resolution
imagery can provide significant value in land use and land cover (LULC) classification. The …

Federated learning (FL) is a challenging setting for optimization due to the heterogeneity of
the data across different clients which gives rise to the client drift phenomenon. In fact …

The vast majority of deep models use multiple gradient signals, typically corresponding to a
sum of multiple loss terms, to update a shared set of trainable weights. However, these …

Deep learning models are increasingly popular in many machine learning applications
where the training data may contain sensitive information. To provide formal and rigorous …

Because learning sometimes involves sensitive data, machine learning algorithms have
been extended to offer differential privacy for training data. In practice, this has been mostly …