blogging doggerel, data, use and misuse - Quants

Once in a great while, I run across of piece of writing I can’t resist. What follows are my poetic summary and the piece itself, which appeared in RealClearPolitics.com and was written by Michael Barone, an inside-the-Beltway political reporter.

Beware of geeks, wonks, and nerds,

They’re apart from common herds.

Now we have to worry about “quants,”

Who think formulas satisfy all wants.

For realists, “quants” are for the birds.

April 16, 2009

On Climate & Health, Beware of Easy Formulas

Michael Barone

Beware of geeks bearing formulas. That's the lesson most of us have learned from the financial crisis. The "quants" who devised the risk models that induced so many financial institutions to buy mortgage-backed securities thought they had reduced risk down to zero.

Turns out they got a few things wrong. Their formulas were based on only a few years of actual data. Or they failed to take into account the possibility that housing prices would fall. Or that the market for mortgage-backed securities might suddenly stop functioning.

The lesson seems clear. Don't allow a whole system to become hostage to the workings of some geek's formula. Keep in mind the possibility that the real world might not behave as the formula indicates.

But, astonishingly, our society seems about to forget that lesson, just as it should have been learned. Congress is poised, at least if the Obama administration gets its way, to pass major new laws on carbon emissions and on health care whose success depends on geeks bearing formulas.

Consider carbon emissions. Carbon dioxide is a harmless gas, not a pollutant. But geeks bearing formulas tell us that increasing amounts of it will heat up the world's climate and cause catastrophic damage some decades hence. Al Gore is so certain of this that he tells us all debate must end -- disagreeing is like denying the Holocaust.

But the Holocaust happened, while the disasters that Gore predicts have not. When you try to predict climate, you are dealing with even more factors and more unknowns than when you try to predict financial risk. Prudent people will want to hedge against some risks that seem possible. But imposing huge costs on the private sector economy -- raising the price of electricity for everybody -- solely on the basis of some geeks' formulas seems, well, not prudent. But that's what Barack Obama tells us we must do.

Or consider health care. One element of proposed health care reforms is restricting care to procedures that are indicated as optimal by "quality metrics." The Obama campaign called for comparative effective research to produce such metrics. The problem, as Dr. Jerome Groopman of Harvard Medical School points out, though not in these words, is that the geeks keep producing different formulas. Or as Dr. Scott Gottlieb of the American Enterprise Institute writes of the comparative effectiveness research mandated in the House-passed stimulus package, "The results of studies are always being made obsolete by new science."

There is a more general problem here. The risk models of the financial geeks, the climate models of the environmental geeks and the medical models of the health care geeks are all ultimately forms of social science. But social science ultimately is not science but art. (italics mine).

The geeks use numbers to try to understand the world. And their work is helpful, up to a point. We can measure the damage to our economy better today than during the 1930s because people then didn't know what the gross national product was. The geeks had not yet invented the formula.

As an avid consumer of political and demographic data since childhood, I am not inclined to say that statistics and formulas are worthless. Rather to the contrary. I grew up knowing that I was one of 1,849,568 people living in Detroit in 1950, and the Census Bureau's estimate that there are only 916,952 people living there today tells you something worth knowing.

But numbers are not reality -- they are just clues (italics mine) In 1965, Daniel Patrick Moynihan, perhaps the most perceptive social scientist of recent times, looked at the numbers of black babies born out of wedlock and predicted a grim future that came to pass. It was not just the numbers, however, but his own experience as a fatherless child that produced this insight.

The financial "quants" failed to realize what a trip to subprime mortgage territory in California's Inland Empire might have told them.

The climate modelers work with historical data that do not necessarily predict future weather patterns. The medical statisticians cannot know the human factors that prompt a sensitive clinician to make lifesaving decisions. Geeks with formulas can help us understand the world better and make informed decisions. But the collapse of our financial institutions tells us that we would be fools to rely on them completely in ordering our great institutions.