In this paper, we present a review of Shannon and differential entropy rate estimation
techniques. Entropy rate, which measures the average information gain from a stochastic …

The data-processing inequality, that is, I (U; Y)≤ I (U; X) for a Markov chain U→ X→ Y, has
been the method of choice for proving impossibility (converse) results in information theory …

The paper considers coding schemes derived from Reed-Muller (RM) codes, for
transmission over input-constrained memoryless channels. Our focus is on the-runlength …

Let 0<; ϵ<; 1/2 be a noise parameter, and let Tϵ be the noise operator acting on functions
on the Boolean cube {0, 1} n. Let f be a nonnegative function on {0, 1} n. We upper bound …

A noisy entropy inequality for boolean functions by Samorodnitsky is applied to binary
codes. It is shown that a binary code that achieves capacity on the binary erasure channel …

The data-processing inequality, that is, $ I (U; Y)\le I (U; X) $ for a Markov chain $ U\to X\to Y
$, has been the method of choice for proving impossibility (converse) results in information …

We propose an information-theoretic Markov aggregation framework that is motivated by two
objectives: 1) The Markov chain observed through the aggregation mapping should be …

This paper studies the capacities of input-driven finite-state channels, ie, channels whose
current state is a time-invariant deterministic function of the previous state and the current …

It has been known for a long time that the mutual information between the input sequence
and output of a binary symmetric channel (BSC) is upper bounded by the mutual information …

This work contains two single-letter upper bounds on the entropy rate of an integer-valued
stationary stochastic process, which only depend on second-order statistics, and are …