Estimation of mutual information from observed samples is a basic primitive in machine
learning, useful in several learning tasks including correlation mining, information …

This paper studies the novel concept of weight correlation in deep neural networks and
discusses its impact on the networks' generalisation ability. For fully-connected layers, the …

Information theory is an outstanding framework for measuring uncertainty, dependence, and
relevance in data and systems. It has several desirable properties for real-world …

Behavior and physiology are essential readouts in many studies but have not benefited from
the high-dimensional data revolution that has transformed molecular and cellular …

The behavior of the Kozachenko–Leonenko estimates for the (differential) Shannon entropy
is studied when the number of iid vector-valued observations tends to infinity. The …

Exogenous state variables and rewards can slow reinforcement learning by injecting
uncontrolled variation into the reward signal. This paper formalizes exogenous state …

Understanding the informative behaviour of deep neural networks is challenged by misused
estimators and the complexity of network structure, which leads to inconsistent observations …

There are numerous different odorant molecules in nature but only a relatively small number
of olfactory receptor neurons (ORNs) in brains. This “compressed sensing” challenge is …

We propose to estimate the KL divergence using a relaxed likelihood ratio estimation in a
Reproducing Kernel Hilbert space. We show that the dual of our ratio estimator for KL in the …

Ecologists frequently seek to establish causal relations between entities of an ecological
system, such as species interactions, ecosystem functions or ecosystem services, using …