[PDF][PDF] Re-evaluating evaluation metrics for spelling checker evaluations
GB Van Huyssteen, ER Eiselen… - Proceedings of First …, 2004 - gerhard.vloek.co.za
GB Van Huyssteen, ER Eiselen, MJ Puttkammer
Proceedings of First Workshop on International Proofing Tools and …, 2004•gerhard.vloek.co.zaIn the increasingly competitive proofing tools market, it is becoming ever more important to
find evaluation methods and metrics that provide stable, invariable measurements. This
article focuses on the evaluation metrics currently used in spelling checker evaluation. We
commence with a brief explication of some presuppositions relevant to the evaluation of
spelling checkers. We then give a detailed description of metrics currently used in various
evaluations of proofing tools, discussing and demonstrating their shortcomings …
find evaluation methods and metrics that provide stable, invariable measurements. This
article focuses on the evaluation metrics currently used in spelling checker evaluation. We
commence with a brief explication of some presuppositions relevant to the evaluation of
spelling checkers. We then give a detailed description of metrics currently used in various
evaluations of proofing tools, discussing and demonstrating their shortcomings …
Abstract
In the increasingly competitive proofing tools market, it is becoming ever more important to find evaluation methods and metrics that provide stable, invariable measurements. This article focuses on the evaluation metrics currently used in spelling checker evaluation. We commence with a brief explication of some presuppositions relevant to the evaluation of spelling checkers. We then give a detailed description of metrics currently used in various evaluations of proofing tools, discussing and demonstrating their shortcomings. Subsequently, we illustrate that, by adjusting some of these metrics, by incorporating additional information (such as the percentage of errors in the text, and a measurement of the suggestion adequacy), and by adapting the evaluation methodology, a single metric can be designed that evaluates the overall linguistic performance of spelling checkers effectively. In conclusion, we suggest how these metrics can be further improved in future research, in order to open up the possibility to set a single benchmark that can be used for any spelling checker, irrespective of the language being evaluated, or the nature of the text used for the evaluation.
gerhard.vloek.co.za
以上显示的是最相近的搜索结果。 查看全部搜索结果