centre for microdata methods and practice

ESRC centre

cemmap is an ESRC research centre


Keep in touch

Subscribe to cemmap news

High-Dimensional Methods and Inference on Structural and Treatment Effects

Authors: Alexandre Belloni , Victor Chernozhukov and Christian Hansen
Date: 03 March 2014
Type: Journal article, Journal of Economic Perspectives, Vol. 8, No.2, pp. 29-50
DOI: 10.1257/jep.28.2.29


Data with a large number of variables relative to the sample size—"high-dimensional data"—are readily available and increasingly common in empirical economics. High-dimensional data arise through a combination of two phenomena. First, the data may be inherently high dimensional in that many different characteristics per observation are available. For example, the US Census collects information on hundreds of individual characteristics and scanner datasets record transaction-level data for households across a wide range of products. Second, even when the number of available variables is relatively small, researchers rarely know the exact functional form with which the small number of variables enter the model of interest. Researchers are thus faced with a large set of potential variables formed by different ways of interacting and transforming the underlying variables. This paper provides an overview of how innovations in "data mining" can be adapted and modified to provide high-quality inference about model parameters. Note that we use the term "data mining" in a modern sense which denotes a principled search for "true" predictive power that guards against false discovery and overfitting, does not erroneously equate in-sample fit to out-of-sample predictive ability, and accurately accounts for using the same data to examine many different hypotheses or models.

Download full version
Previous version:
Alexandre Belloni, Victor Chernozhukov and Christian Hansen November 2013, High dimensional methods and inference on structural and treatment effects, cemmap Working Paper, Cemmap

Search cemmap

Search by title, topic or name.

Contact cemmap

Centre for Microdata Methods and Practice

How to find us

Tel: +44 (0)20 7291 4800

E-mail us