centre for microdata methods and practice

ESRC centre

cemmap is an ESRC research centre

ESRC

Keep in touch

Subscribe to cemmap news

The Quantile Performance of Statistical Treatment Rules Using Hypothesis Tests to Allocate a Population to Two Treatments

Authors: Charles F. Manski and Aleksey Tetenov
Date: 20 November 2014
Type: cemmap Working Paper, CWP44/14
DOI: 10.1920/wp.cem.2014.4414

Abstract

This paper modifies the Wald development of statistical decision theory to offer new perspective on the performance of certain statistical treatment rules.  We study the quantile performance of test rules, ones that use the outcomes of hypothesis tests to allocate a population to two treatments.  Let λ denote the quantile used to evaluate performance.  Define a test rule to be λ-quantile optimal if it maximizes λ-quantile welfare in every state of nature.  We show that a test rule is λ-quantile optimal if and only if its error probabilities are less than λ in all states where the two treatments yield different welfare.  We give conditions under which λ-quantile optimal test rules do and do not exist.  A sufficient condition for existence of optimal rules is that the state space be finite and the data enable sufficiently precise estimation of the true state.  Optimal rules do not exist when the state space is connected and other regularity conditions hold, but near-optimal rules may exist.  These nuanced findings differ sharply from measurement of mean performance, as mean optimal test rules generically do not exist.  We present further analysis that holds when the data are real-valued and generated by a sampling distribution which satisfies the monotone-likelihood ratio (MLR) property with respect to the average treatment effect.  We use the MLR property to characterize the stochastic-dominance admissibility of STRs when the data have a continuous distribution and then generate findings on the quantile admissibility of test rules.

Download full version

Publications feeds

Subscribe to cemmap working papers via RSS

Search cemmap

Search by title, topic or name.

Contact cemmap

Centre for Microdata Methods and Practice

How to find us

Tel: +44 (0)20 7291 4800

E-mail us