Normalization techniques in training dnns: Methodology, analysis and application
Normalization techniques are essential for accelerating the training and improving the
generalization of deep neural networks (DNNs), and have successfully been used in various …
generalization of deep neural networks (DNNs), and have successfully been used in various …
Scalelong: Towards more stable training of diffusion model via scaling network long skip connection
In diffusion models, UNet is the most popular network backbone, since its long skip connects
(LSCs) to connect distant network blocks can aggregate long-distant information and …
(LSCs) to connect distant network blocks can aggregate long-distant information and …
Representative batch normalization with feature calibration
Abstract Batch Normalization (BatchNorm) has become the default component in modern
neural networks to stabilize training. In BatchNorm, centering and scaling operations, along …
neural networks to stabilize training. In BatchNorm, centering and scaling operations, along …
Understanding self-attention mechanism via dynamical system perspective
The self-attention mechanism (SAM) is widely used in various fields of artificial intelligence
and has successfully boosted the performance of different models. However, current …
and has successfully boosted the performance of different models. However, current …
Crossnorm and selfnorm for generalization under distribution shifts
Traditional normalization techniques (eg, Batch Normalization and Instance Normalization)
generally and simplistically assume that training and test data follow the same distribution …
generally and simplistically assume that training and test data follow the same distribution …
A generic shared attention mechanism for various backbone neural networks
The self-attention mechanism is crucial for enhancing various backbone neural networks'
performance. However, current methods add self-attention modules (SAMs) to each network …
performance. However, current methods add self-attention modules (SAMs) to each network …
Rethinking the pruning criteria for convolutional neural network
Channel pruning is a popular technique for compressing convolutional neural networks
(CNNs), where various pruning criteria have been proposed to remove the redundant filters …
(CNNs), where various pruning criteria have been proposed to remove the redundant filters …
Scale region recognition network for object counting in intelligent transportation system
Self-driving technology and safety monitoring devices in intelligent transportation systems
require superb capacity for context awareness. Accurately inferring the counts of crowds and …
require superb capacity for context awareness. Accurately inferring the counts of crowds and …
Re-thinking the effectiveness of batch normalization and beyond
Batch normalization (BN) is used by default in many modern deep neural networks due to its
effectiveness in accelerating training convergence and boosting inference performance …
effectiveness in accelerating training convergence and boosting inference performance …
On fast simulation of dynamical system with neural vector enhanced numerical solver
The large-scale simulation of dynamical systems is critical in numerous scientific and
engineering disciplines. However, traditional numerical solvers are limited by the choice of …
engineering disciplines. However, traditional numerical solvers are limited by the choice of …