Because most philosophies that frown on reproduction don't survive.

Tuesday, October 07, 2008

The Predictions Game

I deal with data. Much of it has to do with looking at sales over the last 3-12 months, and then making predictions as to how many units of a product we would sell at a particular price point. This is predictable up to a point. A product which has never responded to discounting before probably won't respond to it next time. And one that has always done so before will probably respond again.

However making predictions about the future is vastly more inexact than analyzing what went on in the past -- and a lot of people seem to have difficulty dealing with that ambiguity. When looking at an upcoming sale, I might give a sales estimate with a huge amount of deviation: "We'll sell 2000 to 3000 of those cameras at $199." When looking back, it's easy to say, "Well, sales were soft because Circuit City came in with a more aggressive offer; macro economic scares had people reluctant to make major purchases; and we're headed into the season trough where people but off purchases until the holiday season." Is any of true? Probably to one extent or another, but none of it is stuff that is easily to factor in reliably before hand. (We have models for the seasonality, the rest is after the fact guess work.)

This has been striking me in particular as I read about the global financial problems. Financial columnists cheerfully explain what the reasons for a sell off or recovery were, as if these were some sort of easily analyzed or predicted system. But really, these are only explanations that can be applied after the fact. They're stand-ins for the multitude or reasons that might have caused individual people and fund managers to sell particular assets are particular times.

There are broad themes to a set of events made up of millions of unknowable individual decisions. People experience fear and uncertainty and so they're reluctant to lend others money and eager to divest themselves of assets they believe will fall farther in value. But within those themes are uncountable and unknowable numbers of individual decisions that we really can't, as analysts, know much about.


Anonymous said...

I think it was statistician George Box who said, "All models are wrong, but some models are useful."

If you've ever projected a regression line, you see how your confidence interval starts looking like a Venturi tube as you get farther away from your explanatory variables. It doesn't inspire much confidence, to say the least.

I do some simulation modeling in my research, and one of the things that strikes me is how easy it is to adjust the parameters and get a different result. Mind you, I'm simulating relatively stable, predictable things like personnel flows in the military. Imagine trying to do the same thing with highly dynamic and complex flows, like... say... the global climate.

I'm just sayin'...

DMinor said...

Whenever you have a large set of variable data, you have to simplify it into a model, or be overwhelmed into paralysis. What you hope when you make such a model is that your simplification takes into account enough of the important stuff while glossing over mostly trivia.

It always seemed to me that the one thing you could be certain of about economic predictions is that they are always wrong. Useful economic predictions are only a little wrong, while lousy ones are way off the mark. But all are flawed and fall short of perfection (to paraphrase a bit).

On a tangent re the financial crisis, check out a Physics Buzz article on a claim of how physicists and mathmaticians are "to blame" for our current financial predicament.