Advances in computing technology have fostered the development of new and powerful
deep learning (DL) techniques, which have demonstrated promising results in a wide range …

Normalization techniques are essential for accelerating the training and improving the
generalization of deep neural networks (DNNs), and have successfully been used in various …

Many large-scale machine learning (ML) applications need to perform decentralized
learning over datasets generated at different devices and locations. Such datasets pose a …

From the autonomous car driving to medical diagnosis, the requirement of the task of image
segmentation is everywhere. Segmentation of an image is one of the indispensable tasks in …

Abstract Normalization layers (eg, Batch Normalization, Layer Normalization) were
introduced to help with optimization difficulties in very deep nets, but they clearly also help …

Layer normalization (LayerNorm) is a technique to normalize the distributions of
intermediate layers. It enables smoother gradients, faster training, and better generalization …

We showcase important features of the dynamics of the Stochastic Gradient Descent (SGD)
in the training of neural networks. We present empirical observations that commonly used …

Diabetic Retinopathy (DR) is a complication of diabetes mellitus that damages blood vessel
networks in the retina. This is a serious vision-threatening issue in most diabetic subjects …

Abstract Domain adaptation enables knowledge transfer from a labeled source domain to an
unlabeled target domain. A mainstream approach is adversarial feature adaptation, which …

In recent years, larger and deeper models are springing up and continuously pushing state-
of-the-art (SOTA) results across various fields like natural language processing (NLP) and …