Paper #1807

Title:

Non-standard errors

Authors:

Albert J. Menkveld, Anna Dreber, Felix Holzmeister, Juergen Huber, Magnus Johannesson, Michael Kirchler, Sebastian Neussüs, Michael Razen, Utz Weitzel, Christian T. Brownlees, Javier Gil-Bazo and et al.

Date:

December 2021

Abstract:

In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in sample estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.

Keywords:

non-standard errors, multi-analyst approach, liquidity

JEL codes:

C12, C18, G1, G14

Area of Research:

Finance and Accounting

Download the paper in PDF format (1,290 Kb)