We propose new discrimination methods for classification of high dimension, low sample size (HDLSS) data that regularize the degree of data piling. The within-class scatter of the HDLSS data, when projected onto a low-dimensional discriminant subspace, can be selected to be arbitrarily small. Using this fact, we develop two different ways of tuning the amount of within-class scatter, or equivalently, the degree of data piling. In the first approach, we consider a linear path connecting the maximal data piling and the least data piling directions. We also formulate a problem of finding the optimal classifier under a constraint on data piling. The data piling regularization methods are extended to multicategory problems. Simulated and real data examples show competitive performances of the proposed classification methods. Supplementary materials for this article are available online on the journal web site.