Learning to rank has become a popular research topic in several areas such as information retrieval and machine learning. Pair-wise ranking, which learns all the order preferences between pairs of examples, is a typical method for solving the ranking problem. In pair-wise ranking, Rank SVM is a widely-used algorithm and has been successfully applied to the ranking problem in the previous work. However, Rank SVM suffers from the critical problem of long training time needed to deal with a huge number of pairs. In this paper, we propose a data selection technique, Pruned Rank SVM, that selects the most informative pairs before training. Experimental results show that the performance of Pruned Rank SVM is on par with Rank SVM while using significantly fewer pairs.