Multi-armed bandits a simple but very powerful framework for algorithms that make
decisions over time under uncertainty. An enormous body of work has accumulated over the …

We consider online allocation problems with concave revenue functions and resource
constraints, which are central problems in revenue management and online advertising. In …

Multi-armed bandit problems are the predominant theoretical model of exploration-
exploitation tradeoffs in learning, and they have countless applications ranging from medical …

Matching is a classic problem with a rich history and a significant impact, both on the theory
of algorithms and in practice. Recently there has been a surge of interest in the online …

We explore the problem of assigning heterogeneous tasks to workers with different,
unknown skill sets in crowdsourcing markets such as Amazon Mechanical Turk. We first …

The real-time bidding (RTB), aka programmatic buying, has recently become the fastest
growing area in online advertising. Instead of bulking buying and inventory-centric buying …

A natural optimization model that formulates many online resource allocation problems is
the online linear programming (LP) problem in which the constraint matrix is revealed …

Motivated by the availability of real-time data on customer characteristics, we consider the
problem of personalizing the assortment of products for each arriving customer. Using actual …

In this paper, we consider a very general model for exploration-exploitation tradeoff which
allows arbitrary concave rewards and convex constraints on the decisions across time, in …

We consider Bandits with Knapsacks (henceforth, BwK), a general model for multi-armed
bandits under supply/budget constraints. In particular, a bandit algorithm needs to solve a …