Leave deciders alone (7 June 2009)

Economics blogger Matthew Rognile pinpoints what is bothersome about Dan Ariely's Predictably Irrational and the extrapolations he makes from the slew of ingenious studies he details in the book.

The more general philosophical issue here is the tradeoff between internal and external validity. If you're concerned about internal validity, Ariely's work is great. Small sample size notwithstanding, I have very little doubt that if I set up an identical experiment measuring the effects of bonuses on laboratory tasks in India, my results will be similar to Ariely's, and that if prodding lab subjects to perform contrived tasks ever becomes a critical policy goal, this knowledge will prove predictive and invaluable. In this limited sense, I have far more confidence in randomized economic experiments than I do in, say, the correctness of a particular regression specification.

Unfortunately, we are also concerned about external validity—whether our results extend to a more realistic setting—and here we are forced to indulge massive leaps in analysis.

This seems such an obvious problem -- that people act differently in lab studies than in the course of their ordinary lives -- but it also seems that the sorts of clever and pleasing conclusions Ariely typically draws are hard to resist and function well as story or conversation hooks. I'm wary of elevating the idea of revealed preference to the end-all and be-all of studies of decisionmaking; there are too many variables in play to read to much into a fait accopmli decision. But isolating the decision-making process artificially and attempting to control the variables would seem to yield equally limited results. I have the same skepticism about the neurological-scan based studies that Jonah Lehrer details in How We Decide.

Maybe I'm just creeped out more and more by the attempt to reduce decisionmaking to an object of exact science, so that human responses can be better predicted, and inevitably, better programmed in advance.