The Collaboratory for the Study of Earthquake Predictability (CSEP) is an international partnership to support research on rigorous earthquake prediction in multiple tectonic environments. This paper outlines the first earthquake forecast testing experiment for the Japan area conducted within the CSEP framework. We begin with some background and briefly describe efforts in setting up the experiment. The experiment, which closely follows CSEP concepts, is of a prospective sort and is highly objective. Its major feature consists in using Japan, one of the most seismically active and well-instrumented regions in the world, as a natural laboratory. To make full use of this location and of the earthquake catalog maintained by the Japan Meteorological Agency, rules for this experiment have been set up. The experiment consists of 12 categories, with four testing classes each with different time spans (1 day, 3 months, 1 year, and 3 years, respectively) and three testing regions called “All Japan,” “Mainland,” and “Kanto.” A total of 91 models were submitted; these are currently under the CSEP official suite of tests for evaluating the performance of forecasts. This paper briefly describes each model but does not attempt to pass judgment on individual models. Comparative appraisal of the different models will be presented in future publications. Moreover, this is only the first experiment, and more trials are forthcoming. Our aim is to describe what has turned out to be the first occasion for setting up a research environment for rigorous earthquake forecasting in Japan. We argue that now is the time to invest considerably more efforts in related research fields.