Replication Data for: Comparing Experimental and Nonexperimental Methods: What Lessons Have We Learned Four Decades After LaLonde (1986)?

Principal Investigator(s): View help for Principal Investigator(s) Guido Imbens, Stanford University; Yiqing Xu, Stanford University

Version: View help for Version V1

Name	File Type	Size	Last Modified
replication			09/06/2025 05:01:AM

Project Citation:

Imbens, Guido, and Xu, Yiqing. Replication Data for: Comparing Experimental and Nonexperimental Methods: What Lessons Have We Learned Four Decades After LaLonde (1986)? Nashville, TN: American Economic Association [publisher], 2025. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2025-12-22. https://doi.org/10.3886/E232201V1

Project Description

Summary: In 1986, Robert LaLonde published an article comparing nonexperimental estimates to experimental benchmarks (LaLonde 1986). He concluded that the nonexperimental methods at the time could not systematically replicate experimental benchmarks, casting doubt on their credibility. Following LaLonde's critical assessment, there have been significant methodological advances and practical changes, including (i) an emphasis on the unconfoundedness assumption separated from functional form considerations, (ii) a focus on the importance of overlap in covariate distributions, (iii) the introduction of propensity score-based methods leading to doubly robust estimators, (iv) methods for estimating and exploiting treatment effect heterogeneity, and (v) a greater emphasis on validation exercises to bolster research credibility. To demonstrate the practical lessons from these advances, we reexamine the LaLonde data. We show that modern methods, when applied in contexts with sufficient covariate overlap, yield robust estimates for the adjusted differences between the treatment and control groups. However, this does not imply that these estimates are causally interpretable. To assess their credibility, validation exercises (such as placebo tests) are essential, whereas goodness-of-fit tests alone are inadequate. Our findings highlight the importance of closely examining the assignment process, carefully inspecting overlap, and conducting validation exercises when analyzing causal effects with nonexperimental data.

Funding Sources: The Office of Naval Research ( N00014-17-1-2131); The Office of Naval Research (N00014-19-1-2468)

Scope of Project

Subject Terms: causal inference; unconfoundedness

JEL Classification:
C21 Single Equation Models; Single Variables: Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions

Data Type(s): experimental data; observational data

Collection Notes: The data we use in this paper are publicly available.

Methodology

Data Source:

Dehejia and Wahba (1999): https://users.nber.org/~rdehejia/data/.nswdata2.html
Calónico and Smith (2017): https://www.journals.uchicago.edu/doi/abs/10.1086/692397
Imbens, Rubin, and Sacerdote (2001): provided by the first author.

Related Publications

Download this project

Published Versions

V1 [2025-12-22]

Export Metadata

Dublin Core

DDI 2.5

Report a Problem

Found a serious problem with the data, such as disclosure risk or copyrighted content? Let us know.

This material is distributed exactly as received from the data depositor. As of April 2026, depositors are required to submit study materials in accessible formats. ICPSR has not reviewed, checked, or processed this material. For additional information about the study, please contact the investigator(s) directly. If you have questions about the accessibility of materials distributed by ICPSR or require further assistance, please visit ICPSR's Accessibility Center.