Random features for kernel approximation: A survey on algorithms, theory, and beyond

F Liu, X Huang, Y Chen… - IEEE Transactions on …, 2021 - ieeexplore.ieee.org
The class of random features is one of the most popular techniques to speed up kernel
methods in large-scale problems. Related works have been recognized by the NeurIPS Test …

Generalisation error in learning with random features and the hidden manifold model

F Gerace, B Loureiro, F Krzakala… - International …, 2020 - proceedings.mlr.press
We study generalised linear regression and classification for a synthetically generated
dataset encompassing different problems of interest, such as learning with random features …

The unreasonable effectiveness of structured random orthogonal embeddings

KM Choromanski, M Rowland… - Advances in neural …, 2017 - proceedings.neurips.cc
We examine a class of embeddings based on structured random matrices with orthogonal
rows which can be applied in many machine learning applications including dimensionality …

Simplex random features

I Reid, KM Choromanski… - International …, 2023 - proceedings.mlr.press
Abstract We present Simplex Random Features (SimRFs), a new random feature (RF)
mechanism for unbiased approximation of the softmax and Gaussian kernels by geometrical …

On the expressive power of self-attention matrices

V Likhosherstov, K Choromanski, A Weller - arXiv preprint arXiv …, 2021 - arxiv.org
Transformer networks are able to capture patterns in data coming from many domains (text,
images, videos, proteins, etc.) with little or no change to architecture components. We …

The geometry of random features

K Choromanski, M Rowland, T Sarlós… - International …, 2018 - proceedings.mlr.press
We present an in-depth examination of the effectiveness of radial basis function kernel
(beyond Gaussian) estimators based on orthogonal random feature maps. We show that …

Statistical learning guarantees for compressive clustering and compressive mixture modeling

R Gribonval, G Blanchard, N Keriven… - arXiv preprint arXiv …, 2020 - arxiv.org
We provide statistical learning guarantees for two unsupervised learning tasks in the context
of compressive statistical learning, a general framework for resource-efficient large-scale …

Local group invariant representations via orbit embeddings

A Raj, A Kumar, Y Mroueh, T Fletcher… - Artificial Intelligence …, 2017 - proceedings.mlr.press
Invariance to nuisance transformations is one of the desirable properties of effective
representations. We consider transformations that form a group and propose an approach …

High-dimensional similarity search and sketching: algorithms and hardness

I Razenshteyn - 2017 - dspace.mit.edu
We study two fundamental problems that involve massive high-dimensional datasets:
approximate near neighbor search (ANN) and sketching. We obtain a number of new results …

Orlicz random Fourier features

L Chamakh, E Gobet, Z Szabó - Journal of Machine Learning Research, 2020 - jmlr.org
Kernel techniques are among the most widely-applied and inuential tools in machine
learning with applications at virtually all areas of the field. To combine this expressive power …