[HTML][HTML] Manipulation and peer mechanisms: A survey

M Olckers, T Walsh - Artificial Intelligence, 2024 - Elsevier
In peer mechanisms, the competitors for a prize also determine who wins. Each competitor
may be asked to rank, grade, or nominate peers for the prize. Since the prize can be …

On the supervision of peer assessment tasks: an efficient instructor guidance technique

J Hernández-González… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
In peer assessment, students assess a task done by their peers, provide feedback and
usually a grade. The extent to which these peer grades can be used to formally grade the …

Measurement integrity in peer prediction: A peer assessment case study

N Burrell, G Schoenebeck - Proceedings of the 24th ACM Conference on …, 2023 - dl.acm.org
We propose measurement integrity, a property related to ex post reward fairness, as a novel
desideratum for peer prediction mechanisms in many natural applications. Like robustness …

AI-powered methods for academic assessment: overcoming scalability challenges in large university classrooms and conference review

H Zarkoob - 2024 - open.library.ubc.ca
In this thesis, we use various AI techniques to address several scalability challenges in two
academic environments: large university classrooms and large peer-review conferences. In …

Mechanical TA 2: Peer Grading With TA and Algorithmic Support

H Zarkoob, K Leyton-Brown - Proceedings of the 55th ACM Technical …, 2024 - dl.acm.org
Mechanical TA 2 (MTA2) is an open-source, distributed peer grading system that boosts
performance by leveraging both trusted TAs and computationally intensive algorithms. The …

Learning in Teams: Peer Evaluation for Fair Assessment of Individual Contributions

F Duzhin - ECAI 2023, 2023 - ebooks.iospress.nl
We develop a game-theoretical model of a classroom scenario, where n students
collaborate on a common task. We assume that there exists an objective truth known to the …

Simulation-based Approaches for Evaluating Information Elicitation Mechanisms and Information Aggregation Algorithms

N Burrell - 2023 - deepblue.lib.umich.edu
The mathematical study of information elicitation has led to elegant theories about the
behavior of economic agents asked to share their private information. Similarly, the study of …

Learning in teams: peer evaluation for fair assessment of individual contributions

F Duzhin - arXiv preprint arXiv:1904.03333, 2019 - arxiv.org
The" free rider" problem has long plagued pedagogies based on collaborative learning. The
most common solution to the free rider problem is peer evaluation. As well other existing …

Removing Bias and Incentivizing Precision in Peer-grading

A Chakraborty, J Jindal, S Nath - Journal of Artificial Intelligence Research, 2024 - jair.org
Most peer-evaluation practices rely on the evaluator's goodwill and model them as
potentially noisy evaluators. But what if graders are competitive, ie, enjoy higher utility when …