Numerical reasoning in machine reading comprehension tasks: are we there yet?
arXiv preprint arXiv:2109.08207, 2021•arxiv.org
Numerical reasoning based machine reading comprehension is a task that involves reading
comprehension along with using arithmetic operations such as addition, subtraction, sorting,
and counting. The DROP benchmark (Dua et al., 2019) is a recent dataset that has inspired
the design of NLP models aimed at solving this task. The current standings of these models
in the DROP leaderboard, over standard metrics, suggest that the models have achieved
near-human performance. However, does this mean that these models have learned to …
comprehension along with using arithmetic operations such as addition, subtraction, sorting,
and counting. The DROP benchmark (Dua et al., 2019) is a recent dataset that has inspired
the design of NLP models aimed at solving this task. The current standings of these models
in the DROP leaderboard, over standard metrics, suggest that the models have achieved
near-human performance. However, does this mean that these models have learned to …
Numerical reasoning based machine reading comprehension is a task that involves reading comprehension along with using arithmetic operations such as addition, subtraction, sorting, and counting. The DROP benchmark (Dua et al., 2019) is a recent dataset that has inspired the design of NLP models aimed at solving this task. The current standings of these models in the DROP leaderboard, over standard metrics, suggest that the models have achieved near-human performance. However, does this mean that these models have learned to reason? In this paper, we present a controlled study on some of the top-performing model architectures for the task of numerical reasoning. Our observations suggest that the standard metrics are incapable of measuring progress towards such tasks.
arxiv.org
以上显示的是最相近的搜索结果。 查看全部搜索结果