We review the principal information theoretic tools and their use for feature selection, with
the main emphasis on classification problems with discrete features. Since it is known that …

We investigate the analogy between the renormalization group (RG) and deep neural
networks, wherein subsequent layers of neurons are analogous to successive steps along …

Learning fair and transferable representations of users that can be used for a wide spectrum
of downstream tasks (specifically, machine learning models) has great potential in fairness …

Calculating the capacity (with or without feedback) of channels with memory and continuous
alphabets is a challenging task. It requires optimizing the directed information (DI) rate over …

Correlations between factors of variation are prevalent in real-world data. Exploiting such
correlations may increase predictive performance on noisy data; however, often correlations …

Reinforcement learning (RL) problems where the learner attempts to infer an unobserved
reward from some feedback variables have been studied in several recent papers. The …

Novel approaches to estimate information measures using neural networks are well-
celebrated in recent years both in the information theory and machine learning communities …

This letter considers a data-driven approach to a secret key rate analysis of physical layer
secret key generation based on mutual information neural estimator (MINE), which achieves …

The estimation of mutual information (MI) or conditional mutual information (CMI) from a set
of samples is a longstanding problem. A recent line of work in this area has leveraged the …

The present paper aims to propose a new method for neutralizing contradictions in neural
networks. Neural networks exhibit numerous contradictions in the form of contrasts …