Peter Rossi's Metallic Laws of Policy Evaluation

In an article by Jens Ludwig, Jeffrey R. Kling and Sendhil Mullainathan in the Summer 2011 issue of my own Journal of Economic Perspectives, called "Mechanism Experiments and Policy Evaluations," they introduced me to the "metallic rules" of the eminent sociologist Peter H. Rossi. The rules come from Rossi's 1987 article,“The Iron Law of Evaluation and Other Metallic Rules,” Research in Social Problems and Public Policy, vol. 4, pp. 3–20. The article is pre-web, and the publication is not easily available, but David Roodman at the Center for Global Development tracked down the article a couple of years ago, and posted an excerpt summarizing the rules on his Microfinance Open Book Blog. Rossi died in 2006; a brief obituary and some remembrances from colleagues are available at the website of the American Sociological Association.


Here are Peter Rossi's "metallic rules" of policy evaluation from his 1987 article: 

A dramatic but slightly overdrawn view of two decades of evaluation efforts can be stated as a set of “laws,” each summarizing some strong tendency that can be discerned in that body of materials. Following a 19th Century practice that has fallen into disuse in social science, these laws are named after substances of varying durability, roughly indexing each law’s robustness. 

The Iron Law of Evaluation: The expected value of any net impact assessment of any large scale social program is zero.
The Iron Law arises from the experience that few impact assessments of large scale social programs have found that the programs in question had any net impact. The law also means that, based on the evaluation efforts of the last twenty years, the best a priori estimate of the net impact assessment of any program is zero, i.e., that the program will have no effect.

The Stainless Steel Law of Evaluation: The better designed the impact assessment of a social program, the more likely is the resulting estimate of net impact to be zero.
This law means that the more technically rigorous the net impact assessment, the more likely are its results to be zero—or no effect. Specifically, this law implies that estimating net impacts through randomized controlled experiments, the avowedly best approach to estimating net impacts, is more likely to show zero effects than other less rigorous approaches. 

The Brass Law of Evaluation: The more social programs are designed to change individuals, the more likely the net impact of the program will be zero.
This law means that social programs designed to rehabilitate individuals by changing them in some way or another are more likely to fail. The Brass Law may appear to be redundant since all programs, including those designed to deal with individuals, are covered by the Iron Law. This redundancy is intended to emphasize the especially difficult task faced in designing and implementing effective programs that are designed to rehabilitate individuals. 

The Zinc Law of Evaluation: Only those programs that are likely to fail are evaluated.
Of the several metallic laws of evaluation, the zinc law has the most optimistic slant since it implies that there are effective programs but that such effective programs are never evaluated. It also implies that if a social program is effective, that characteristic is obvious enough and hence policy makers and others who sponsor and fund evaluations decide against evaluation.