Conventionally, it is often assumed that the training and testing data distributions are the same and that all classes in the test set are observed in the training set. However, this assumption may not be true in real-world tasks. In practice, there may be test samples from classes that were unknown during training. In such cases, one would like the adopted model to have the capacity to classify such samples into a "none of the above" class. This task is known as open-set recognition, and it has gained significant attention in recent years. In this paper, we propose a novel distance-based open-set recognition approach by constructing task-specific distance metrics with Gaussian processes. Our experimental results demonstrate that the proposed approach outperforms state-of-the-art methods on both synthetic and real-world datasets.