A newer version of this project is available. See below for other available versions.

ECIN Replication Package for "Is Economics self-correcting? Replications in the American Economic Review"

Name: ECIN Replication Package for "Is Economics self-correcting? Replications in the American Economic Review"
Published: 2024-02-24
License: https://creativecommons.org/licenses/by/4.0

Principal Investigator(s): View help for Principal Investigator(s) Jörg Ankel-Peters, RWI - Leibniz Institute for Economic Research; Nathan Fiala, University of Connecticut; Florian Neubauer, University of Connecticut and RWI - Leibniz Institute for Economic Research

Version: View help for Version V2

Name	File Type	Size	Last Modified
01_DoFiles			02/24/2024 05:52:PM
02_Data			02/24/2024 05:53:PM
Readme.pdf	application/pdf	108.4 KB	02/24/2024 12:52:PM

Project Citation:

Ankel-Peters, Jörg , Fiala, Nathan, and Neubauer, Florian. ECIN Replication Package for “Is Economics self-correcting? Replications in the American Economic Review.” Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2024-02-24. https://doi.org/10.3886/E198622V2

Project Description

Summary: This paper reviews the impact of replications published as comments in the American Economic Review between 2010 and 2020. We examine their citations and influence on the original papers’ subsequent citations. Our results show that comments are barely cited, and they do not affect the original paper’s citations – even if the comment diagnoses substantive problems. Furthermore, we conduct an opinion survey among replicators and authors and find that there often is no consensus on whether the original paper’s contribution sustains. We conclude that the economics literature does not self-correct, and that robustness and replicability are hard to define in economics.

Funding Sources: Deutsche Forschungsgemeinschaft, DFG (3473/1-1)

Scope of Project

Subject Terms: Replication; citations; meta-science

JEL Classification:
      A11 Role of Economics; Role of Economists; Market for Economists
      A14 Sociology of Economics
      B40 Economic Methodology: General

Manuscript Number: ECIN-Feb-2023-0095.R2

Geographic Coverage: worldwide

Time Period(s): 1980 – 2021

Universe: Comments published in the AER between 2010 and 2020, the original papers they are written on, the authors of comments and original papers, the editors of the AER, and all papers and comments published between 1980 and 2021 in the AER.

Data Type(s): observational data; survey data

Methodology

Response Rate: 57% of authors responded to our author survey.
100% of living AER editors responded to our editor survey.

Sampling: We include all comments in the AER between 2010 and 2020 as well as the original papers they are written on in our sample.

Data Source: The American Economic Review, American Economic Association. Accessed at: https://www.aeaweb.org/journals/aer

Collection Mode(s): mail questionnaire; other; web-based survey

Scales: A likert-type scale was used for most questions in the author survey. We also included open-ended questions, as well as asking for numbers.
The editor survey consisted of open ended questions via email.
The citation data was manually collected from GoogleScholar.

Unit(s) of Observation: Papers, Individuals.

Related Publications

Download this project

Published Versions

Export Metadata

Dublin Core

DDI 2.5

Report a Problem

Found a serious problem with the data, such as disclosure risk or copyrighted content? Let us know.

This material is distributed exactly as it arrived from the data depositor. ICPSR has not checked or processed this material. Users should consult the investigator(s) if further information is desired.