SemEval 2021 task 7: HaHackathon, detecting and rating humor and offense

JA Meaney, S Wilson, L Chiruzzo… - Proceedings of the …, 2021 - aclanthology.org
Proceedings of the 15th International Workshop on Semantic Evaluation …, 2021aclanthology.org
Abstract SemEval 2021 Task 7, HaHackathon, was the first shared task to combine the
previously separate domains of humor detection and offense detection. We collected 10,000
texts from Twitter and the Kaggle Short Jokes dataset, and had each annotated for humor
and offense by 20 annotators aged 18-70. Our subtasks were binary humor detection,
prediction of humor and offense ratings, and a novel controversy task: to predict if the
variance in the humor ratings was higher than a specific threshold. The subtasks attracted …
Abstract
SemEval 2021 Task 7, HaHackathon, was the first shared task to combine the previously separate domains of humor detection and offense detection. We collected 10,000 texts from Twitter and the Kaggle Short Jokes dataset, and had each annotated for humor and offense by 20 annotators aged 18-70. Our subtasks were binary humor detection, prediction of humor and offense ratings, and a novel controversy task: to predict if the variance in the humor ratings was higher than a specific threshold. The subtasks attracted 36-58 submissions, with most of the participants choosing to use pre-trained language models. Many of the highest performing teams also implemented additional optimization techniques, including task-adaptive training and adversarial training. The results suggest that the participating systems are well suited to humor detection, but that humor controversy is a more challenging task. We discuss which models excel in this task, which auxiliary techniques boost their performance, and analyze the errors which were not captured by the best systems.
aclanthology.org
以上显示的是最相近的搜索结果。 查看全部搜索结果