Wikibench: Community-driven data curation for ai evaluation on wikipedia

TS Kuo, AL Halfaker, Z Cheng, J Kim, MH Wu… - Proceedings of the CHI …, 2024 - dl.acm.org
AI tools are increasingly deployed in community contexts. However, datasets used to
evaluate AI are typically created by developers and annotators outside a given community …

A deep active learning-based and crowdsourcing-assisted solution for named entity recognition in Chinese historical corpora

C Yan, X Tang, H Yang, J Wang - Aslib Journal of Information …, 2023 - emerald.com
Purpose The majority of existing studies about named entity recognition (NER) concentrate
on the prediction enhancement of deep neural network (DNN)-based models themselves …

Crowdsourcing subjective annotations using pairwise comparisons reduces bias and error compared to the majority-vote method

H Narimanzadeh, A Badie-Modiri, IG Smirnova… - Proceedings of the …, 2023 - dl.acm.org
How to better reduce measurement variability and bias introduced by subjectivity in
crowdsourced labelling remains an open question. We introduce a theoretical framework for …

Learning from biased crowdsourced labeling with deep clustering

M Wu, Q Li, F Yang, J Zhang, VS Sheng… - Expert Systems with …, 2023 - Elsevier
With the rapid development of crowdsourcing learning, amount of labels can be obtained
from crowd workers fast and cheaply. However, crowdsourcing learning also faces …

LLMs as Research Tools: Applications and Evaluations in HCI Data Work

M Aubin Le Quéré, H Schroeder, C Randazzo… - Extended Abstracts of …, 2024 - dl.acm.org
Large language models (LLMs) stand to reshape traditional methods of working with data.
While LLMs unlock new and potentially useful ways of interfacing with data, their use in …

Learning from Time Series under Temporal Label Noise

S Nagaraj, W Gerych, S Tonekaboni… - arXiv preprint arXiv …, 2024 - arxiv.org
Many sequential classification tasks are affected by label noise that varies over time. Such
noise can cause label quality to improve, worsen, or periodically change over time. We first …

Learning from Crowds Using Graph Neural Networks with Attention Mechanism

J Zhang, M Wu, Z Sun, C Zhou - IEEE Transactions on Big Data, 2024 - ieeexplore.ieee.org
Crowdsourcing has been playing an essential role in machine learning since it can obtain a
large number of labels in an economical and fast manner for training increasingly complex …

Automated ISAR Image Quality Assessment

T Jasinski, L Rosenberg… - 2023 IEEE International …, 2023 - ieeexplore.ieee.org
Inverse Synthetic Aperture Radar (ISAR) is a widespread imaging technology used to
characterise and classify non-cooperative targets. Unfortunately, the quality of ISAR imagery …

Learning with Temporal Label Noise

Many sequential classification tasks are affected by label noise that changes over time. Such
noise might arise from label quality improving, worsening, or periodically changing over …