Facet: Fairness in computer vision evaluation benchmark
Computer vision models have known performance disparities across attributes such as
gender and skin tone. This means during tasks such as classification and detection, model …
gender and skin tone. This means during tasks such as classification and detection, model …
A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI
As Artificial Intelligence (AI) continues to advance rapidly, it becomes increasingly important
to consider AI's ethical and societal implications. In this paper, we present a bottom-up …
to consider AI's ethical and societal implications. In this paper, we present a bottom-up …
AI Consent Futures: A Case Study on Voice Data Collection with Clinicians
As new forms of data capture emerge to power new AI applications, questions abound about
the ethical implications of these data collection practices. In this paper, we present clinicians' …
the ethical implications of these data collection practices. In this paper, we present clinicians' …
Who broke Amazon Mechanical Turk? An analysis of crowdsourcing data quality over time
CC Marshall, PSR Goguladinne… - Proceedings of the 15th …, 2023 - dl.acm.org
We present the results of a survey fielded in June of 2022 as a lens to examine recent data
reliability issues on Amazon Mechanical Turk. We contrast bad data from this survey with …
reliability issues on Amazon Mechanical Turk. We contrast bad data from this survey with …
Ethical considerations for responsible data curation
Human-centric computer vision (HCCV) data curation practices often neglect privacy and
bias concerns, leading to dataset retractions and unfair models. HCCV datasets constructed …
bias concerns, leading to dataset retractions and unfair models. HCCV datasets constructed …
Wikibench: Community-driven data curation for ai evaluation on wikipedia
AI tools are increasingly deployed in community contexts. However, datasets used to
evaluate AI are typically created by developers and annotators outside a given community …
evaluate AI are typically created by developers and annotators outside a given community …
Studying Collaborative Interactive Machine Teaching in Image Classification
B Mohammadzadeh, J Françoise, M Gouiffès… - Proceedings of the 29th …, 2024 - dl.acm.org
While human-centered approaches to machine learning explore various human roles within
the interaction loop, the notion of Interactive Machine Teaching (IMT) emerged with a focus …
the interaction loop, the notion of Interactive Machine Teaching (IMT) emerged with a focus …
Fairness feedback loops: training on synthetic data amplifies bias
Model-induced distribution shifts (MIDS) occur as previous model outputs pollute new model
training sets over generations of models. This is known as model collapse in the case of …
training sets over generations of models. This is known as model collapse in the case of …
Discipline and Label: A WEIRD Genealogy and Social Theory of Data Annotation
Data annotation remains the sine qua non of machine learning and AI. Recent empirical
work on data annotation has begun to highlight the importance of rater diversity for fairness …
work on data annotation has begun to highlight the importance of rater diversity for fairness …
Towards Bidirectional Human-AI Alignment: A Systematic Review for Clarifications, Framework, and Future Directions
Recent advancements in general-purpose AI have highlighted the importance of guiding AI
systems towards the intended goals, ethical principles, and values of individuals and …
systems towards the intended goals, ethical principles, and values of individuals and …