We present a review on the recent advances and emerging opportunities around the theme
of analyzing deep neural networks (DNNs) with information-theoretic methods. We first …

We consider the problem of improving the adversarial robustness of neural networks while
retaining natural accuracy. Motivated by the multi-view information bottleneck formalism, we …

Despite the empirical success of knowledge distillation, current state-of-the-art methods are
computationally expensive to train, which makes them difficult to adopt in practice. To …

Fine-grained visual categorization (FGVC) aims to discriminate similar subcategories,
whose main challenge is the large intraclass diversities and subtle inter-class differences …

By" intelligently" fuse the complementary information across different views, multi-view
learning is able to improve the performance of classification task. In this work, we extend the …

Deep neural networks suffer from poor generalization to unseen environments when the
underlying data distribution is different from that in the training set. By learning minimum …

Neural networks have dramatically increased our capacity to learn from large, high-
dimensional datasets across innumerable disciplines. However, their decisions are not …

The Matrix-based Rényi's entropy enables us to directly measure information quantities from
given data without the costly probability density estimation of underlying distributions, thus …

Most entropy measures depend on the spread of the probability distribution over the sample
space| X|, and the maximum entropy achievable scales proportionately with the sample …

Understanding the informative behaviour of deep neural networks is challenged by misused
estimators and the complexity of network structure, which leads to inconsistent observations …