Calibrating classification probabilities with shape-restricted polynomial regression
IEEE transactions on pattern analysis and machine intelligence, 2019•ieeexplore.ieee.org
In many real-world classification problems, accurate prediction of membership probabilities
is critical for further decision making. The probability calibration problem studies how to map
scores obtained from one classification algorithm to membership probabilities. The
requirement of non-decreasingness for this mapping involves an infinite number of
inequality constraints, which makes its estimation computationally intractable. For the sake
of this difficulty, existing methods failed to achieve four desiderata of probability calibration …
is critical for further decision making. The probability calibration problem studies how to map
scores obtained from one classification algorithm to membership probabilities. The
requirement of non-decreasingness for this mapping involves an infinite number of
inequality constraints, which makes its estimation computationally intractable. For the sake
of this difficulty, existing methods failed to achieve four desiderata of probability calibration …
In many real-world classification problems, accurate prediction of membership probabilities is critical for further decision making. The probability calibration problem studies how to map scores obtained from one classification algorithm to membership probabilities. The requirement of non-decreasingness for this mapping involves an infinite number of inequality constraints, which makes its estimation computationally intractable. For the sake of this difficulty, existing methods failed to achieve four desiderata of probability calibration: universal flexibility, non-decreasingness, continuousness and computational tractability. This paper proposes a method with shape-restricted polynomial regression, which satisfies all four desiderata. In the method, the calibrating function is approximated with monotone polynomials, and the continuously-constrained requirement of monotonicity is equivalent to some semidefinite constraints. Thus, the calibration problem can be solved with tractable semidefinite programs. This estimator is both strongly and weakly universally consistent under a trivial condition. Experimental results on both artificial and real data sets clearly show that the method can greatly improve calibrating performance in terms of reliability-curve related measures.
ieeexplore.ieee.org
以上显示的是最相近的搜索结果。 查看全部搜索结果