Evaluation on crowdsourcing research: Current status and future direction

Y Zhao, Q Zhu - Information systems frontiers, 2014 - Springer
Crowdsourcing is one of the emerging Web 2.0 based phenomenon and has attracted great
attention from both practitioners and scholars over the years. It can facilitate the connectivity …

Should chatgpt be biased? challenges and risks of bias in large language models

E Ferrara - arXiv preprint arXiv:2304.03738, 2023 - arxiv.org
As the capabilities of generative language models continue to advance, the implications of
biases ingrained within these models have garnered increasing attention from researchers …

Dynabench: Rethinking benchmarking in NLP

D Kiela, M Bartolo, Y Nie, D Kaushik, A Geiger… - arXiv preprint arXiv …, 2021 - arxiv.org
We introduce Dynabench, an open-source platform for dynamic dataset creation and model
benchmarking. Dynabench runs in a web browser and supports human-and-model-in-the …

Megastudies, crowdsourcing, and large datasets in psycholinguistics: An overview of recent developments

E Keuleers, DA Balota - Quarterly Journal of Experimental …, 2015 - journals.sagepub.com
This paper introduces and summarizes the special issue on megastudies, crowdsourcing,
and large datasets in psycholinguistics. We provide a brief historical overview and show …

The CANDOR corpus: Insights from a large multimodal dataset of naturalistic conversation

A Reece, G Cooney, P Bull, C Chung, B Dawson… - Science …, 2023 - science.org
People spend a substantial portion of their lives engaged in conversation, and yet, our
scientific understanding of conversation is still in its infancy. Here, we introduce a large …

Evaluating natural language understanding services for conversational question answering systems

D Braun, AH Mendez, F Matthes… - Proceedings of the 18th …, 2017 - aclanthology.org
Conversational interfaces recently gained a lot of attention. One of the reasons for the
current hype is the fact that chatbots (one particularly popular form of conversational …

[HTML][HTML] Age-of-acquisition ratings for 30,000 English words

V Kuperman, H Stadthagen-Gonzalez… - Behavior research …, 2012 - Springer
We present age-of-acquisition (AoA) ratings for 30,121 English content words (nouns, verbs,
and adjectives). For data collection, this megastudy used the Web-based crowdsourcing …

A computational approach to politeness with application to social factors

C Danescu-Niculescu-Mizil, M Sudhof… - arXiv preprint arXiv …, 2013 - arxiv.org
We propose a computational framework for identifying linguistic aspects of politeness. Our
starting point is a new corpus of requests annotated for politeness, which we use to evaluate …

Self-disclosure and perceived trustworthiness of Airbnb host profiles

X Ma, JT Hancock, K Lim Mingjie… - Proceedings of the 2017 …, 2017 - dl.acm.org
Online peer-to-peer platforms like Airbnb allow hosts to list a property (eg a house, or a
room) for short-term rentals. In this work, we examine how hosts describe themselves on …

[PDF][PDF] Linguistic models for analyzing and detecting biased language

M Recasens, C Danescu-Niculescu-Mizil… - Proceedings of the …, 2013 - aclanthology.org
Unbiased language is a requirement for reference sources like encyclopedias and scientific
texts. Bias is, nonetheless, ubiquitous, making it crucial to understand its nature and …