Statistical physics of inference: Thresholds and algorithms

L Zdeborová, F Krzakala - Advances in Physics, 2016 - Taylor & Francis
Many questions of fundamental interest in today's science can be formulated as inference
problems: some partial, or noisy, observations are performed over a set of variables and the …

Entropy and mutual information in models of deep neural networks

M Gabrié, A Manoel, C Luneau… - Advances in neural …, 2018 - proceedings.neurips.cc
We examine a class of stochastic deep learning models with a tractable method to compute
information-theoretic quantities. Our contributions are three-fold:(i) We show how entropies …

Rigorous dynamics of expectation-propagation-based signal recovery from unitarily invariant measurements

K Takeuchi - IEEE Transactions on Information Theory, 2019 - ieeexplore.ieee.org
Signal recovery from unitarily invariant measurements is investigated in this paper. A
message-passing algorithm is formulated on the basis of expectation propagation (EP). A …

A typical reconstruction limit for compressed sensing based on lp-norm minimization

Y Kabashima, T Wadayama… - Journal of Statistical …, 2009 - iopscience.iop.org
We consider the problem of reconstructing an N-dimensional continuous vector x from P
constraints which are generated from its linear transformation under the assumption that the …

Memory amp

L Liu, S Huang, BM Kurkoski - IEEE Transactions on …, 2022 - ieeexplore.ieee.org
Approximate message passing (AMP) is a low-cost iterative parameter-estimation technique
for certain high-dimensional linear systems with non-Gaussian distributions. AMP only …

Spectral universality of regularized linear regression with nearly deterministic sensing matrices

R Dudeja, S Sen, YM Lu - arXiv preprint arXiv:2208.02753, 2022 - arxiv.org
It has been observed that the performances of many high-dimensional estimation problems
are universal with respect to underlying sensing (or design) matrices. Specifically, matrices …

Bayes-optimal convolutional AMP

K Takeuchi - IEEE Transactions on Information Theory, 2021 - ieeexplore.ieee.org
This paper proposes Bayes-optimal convolutional approximate message-passing (CAMP)
for signal recovery in compressed sensing. CAMP uses the same low-complexity matched …

The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?

J Barbier, TQ Hou, M Mondelli… - Advances in Neural …, 2022 - proceedings.neurips.cc
We consider the problem of estimating a rank-$1 $ signal corrupted by structured rotationally
invariant noise, and address the following question:\emph {how well do inference algorithms …

On the convergence of orthogonal/vector AMP: Long-memory message-passing strategy

K Takeuchi - IEEE Transactions on Information Theory, 2022 - ieeexplore.ieee.org
Orthogonal/vector approximate message-passing (AMP) is a powerful message-passing
(MP) algorithm for signal reconstruction in compressed sensing. This paper proves the …

Compact optical one-way waveguide isolators for photonic-band-gap microchips

H Takeda, S John - Physical Review A—Atomic, Molecular, and Optical …, 2008 - APS
In two-dimensional (2D) photonic crystals (PC's), we demonstrate compact optical
waveguide isolators in which light can propagate only one way. Waveguides are composed …