In this paper, we present a review of Shannon and differential entropy rate estimation
techniques. Entropy rate, which measures the average information gain from a stochastic …

We give a brief survey of the literature on the empirical estimation of entropy, differential
entropy, relative entropy, mutual information and related information measures. While those …

Abstract The mixing time $ t_ {\mathsf {mix}} $ of an ergodic Markov chain measures the rate
of convergence towards its stationary distribution $\boldsymbol {\pi} $. We consider the …

Entropy estimation is one of the prototypical problems in distribution property testing. To
consistently estimate the Shannon entropy of a distribution on $ S $ elements with …

We study the following learning problem with dependent data: Given a trajectory of length $
n $ from a stationary Markov chain with $ k $ states, the goal is to predict the distribution of …

In this paper, we evaluate clock signals generated in ring oscillators and self-timed rings and
the way their jitter can be transformed into random numbers. We show that counting the …

Since the entropy is a popular randomness measure, there are many studies for the
estimation of entropies for given random samples. In this paper, we propose an estimation …

We show through case studies that it is easier to estimate the fundamental limits of data
processing than to construct the explicit algorithms to achieve those limits. Focusing on …

Let be a stationary and ergodic process with joint distribution, where the random variables
take values in a finite set. Let be the first time this process repeats its first n symbols of …

We study the following learning problem with dependent data: Observing a trajectory of
length from a stationary Markov chain with states, the goal is to predict the next state. For …