Random features for kernel approximation: A survey on algorithms, theory, and beyond
The class of random features is one of the most popular techniques to speed up kernel
methods in large-scale problems. Related works have been recognized by the NeurIPS Test …
methods in large-scale problems. Related works have been recognized by the NeurIPS Test …
Generalisation error in learning with random features and the hidden manifold model
We study generalised linear regression and classification for a synthetically generated
dataset encompassing different problems of interest, such as learning with random features …
dataset encompassing different problems of interest, such as learning with random features …
The unreasonable effectiveness of structured random orthogonal embeddings
KM Choromanski, M Rowland… - Advances in neural …, 2017 - proceedings.neurips.cc
We examine a class of embeddings based on structured random matrices with orthogonal
rows which can be applied in many machine learning applications including dimensionality …
rows which can be applied in many machine learning applications including dimensionality …
Simplex random features
I Reid, KM Choromanski… - International …, 2023 - proceedings.mlr.press
Abstract We present Simplex Random Features (SimRFs), a new random feature (RF)
mechanism for unbiased approximation of the softmax and Gaussian kernels by geometrical …
mechanism for unbiased approximation of the softmax and Gaussian kernels by geometrical …
On the expressive power of self-attention matrices
Transformer networks are able to capture patterns in data coming from many domains (text,
images, videos, proteins, etc.) with little or no change to architecture components. We …
images, videos, proteins, etc.) with little or no change to architecture components. We …
The geometry of random features
We present an in-depth examination of the effectiveness of radial basis function kernel
(beyond Gaussian) estimators based on orthogonal random feature maps. We show that …
(beyond Gaussian) estimators based on orthogonal random feature maps. We show that …
Statistical learning guarantees for compressive clustering and compressive mixture modeling
We provide statistical learning guarantees for two unsupervised learning tasks in the context
of compressive statistical learning, a general framework for resource-efficient large-scale …
of compressive statistical learning, a general framework for resource-efficient large-scale …
Local group invariant representations via orbit embeddings
Invariance to nuisance transformations is one of the desirable properties of effective
representations. We consider transformations that form a group and propose an approach …
representations. We consider transformations that form a group and propose an approach …
High-dimensional similarity search and sketching: algorithms and hardness
I Razenshteyn - 2017 - dspace.mit.edu
We study two fundamental problems that involve massive high-dimensional datasets:
approximate near neighbor search (ANN) and sketching. We obtain a number of new results …
approximate near neighbor search (ANN) and sketching. We obtain a number of new results …
Orlicz random Fourier features
Kernel techniques are among the most widely-applied and inuential tools in machine
learning with applications at virtually all areas of the field. To combine this expressive power …
learning with applications at virtually all areas of the field. To combine this expressive power …