Heterogeneity in effect size estimates

F Holzmeister, M Johannesson, R Böhm… - Proceedings of the …, 2024 - pnas.org
A typical empirical study involves choosing a sample, a research design, and an analysis
path. Variation in such choices across studies leads to heterogeneity in results that introduce …

How do data analysts respond to ai assistance? a wizard-of-oz study

K Gu, M Grunde-McLaughlin, A McNutt, J Heer… - Proceedings of the CHI …, 2024 - dl.acm.org
Data analysis is challenging as analysts must navigate nuanced decisions that may yield
divergent conclusions. AI assistants have the potential to support analysts in planning their …

A framework for evaluating reproducibility and replicability in economics

A Dreber, M Johannesson - Economic Inquiry, 2024 - Wiley Online Library
We propose a framework for evaluating reproducibility and replicability in economics.
Reproducibility is defined as testing if the results of an original study can be reproduced …

Do anomalies really predict market returns? New data and new evidence

N Cakici, C Fieberg, D Metko, A Zaremba - Review of Finance, 2024 - academic.oup.com
Using new data from US and global markets, we revisit market risk premium predictability by
equity anomalies. We apply a repertoire of machine-learning methods to forty-two countries …

Reproducibility in Management Science

M Fišar, B Greiner, C Huber, E Katok… - Management …, 2024 - pubsonline.informs.org
With the help of more than 700 reviewers, we assess the reproducibility of nearly 500
articles published in the journal Management Science before and after the introduction of a …

Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

N Breznau, EM Rinke, A Wuttke… - Proceedings of the …, 2022 - National Acad Sciences
This study explores how researchers' analytical choices affect the reliability of scientific
findings. Most discussions of reliability problems in science focus on systematic biases. We …

[HTML][HTML] Non-standard errors in asset pricing: Mind your sorts

A Soebhag, B Van Vliet, P Verwijmeren - Journal of Empirical Finance, 2024 - Elsevier
Non-standard errors capture variation due to differences in research design choices. We
document large variation in design choices in the context of asset pricing factor models and …

Experimenting with financial professionals

C Huber, C König-Kersting - 2022 - econstor.eu
Financial professionals play a key role in financial markets and the financial industry as a
whole. Researchers in experimental economics and finance have therefore started to …

Toward a more credible assessment of the credibility of science by many-analyst studies

K Auspurg, J Brüderl - Proceedings of the National Academy of Sciences, 2024 - pnas.org
We discuss a relatively new meta-scientific research design: many-analyst studies that
attempt to assess the replicability and credibility of research based on large-scale …

Subjective evidence evaluation survey for many-analysts studies

A Sarafoglou, S Hoogeveen… - Royal Society …, 2024 - royalsocietypublishing.org
Many-analysts studies explore how well an empirical claim withstands plausible alternative
analyses of the same dataset by multiple, independent analysis teams. Conclusions from …