Monday, December 16, 2013

Just How Good Are Public Health, Climate Science and Financial Models?

From Aeon Magazine:
A model world
In economics, climate science and public health, computer models help us decide how to act. But can we trust them?
Here’s a simple recipe for doing science. Find a plausible theory for how some bits of the world behave, make predictions, test them experimentally. If the results fit the predictions, then the theory might describe what’s really going on. If not, you need to think again. Scientific work is vastly diverse and full of fascinating complexities. Still, the recipe captures crucial features of how most of it has been done for the past few hundred years.

Now, however, there is a new ingredient. Computer simulation, only a few decades old, is transforming scientific projects as mind-bending as plotting the evolution of the cosmos, and as mundane as predicting traffic snarl-ups. What should we make of this scientific nouvelle cuisine? While it is related to experiment, all the action is in silico — not in the world, or even the lab. It might involve theory, transformed into equations, then computer code. Or it might just incorporate some rough approximations, which are good enough to get by with. Made digestible, the results affect us all.

As computer modelling has become essential to more and more areas of science, it has also become at least a partial guide to headline-grabbing policy issues, from flood control and the conserving of fish stocks, to climate change and — heaven help us — the economy. But do politicians and officials understand the limits of what these models can do? Are they all as good, or as bad, as each other? If not, how can we tell which is which?

Modelling is an old word in science, and the old uses remain. It can mean a way of thinking grounded in analogy — electricity as a fluid that flows, an atom as a miniature solar system. Or it can be more like the child’s toy sense of model: an actual physical model of something that serves as an aid to thought. Recall James Watson in 1953 using first cardboard, then brass templates cut in the shape of the four bases in DNA so that he could shuffle them around and consider how they might fit together in what emerged as the double-helix model of the genetic material.

Computer models are different. They’re often more complex, always more abstract and, crucially, they’re dynamic. It is the dynamics that call for the computation. Somewhere in the model lies an equation or set of equations that represent how some variables are tied to others: change one quantity, and working through the mathematics will tell you how it affects the rest. In most systems, tracking such changes over time quickly overwhelms human powers of calculation. But with today’s super-fast computers, such dynamic problems are becoming soluble. Just turn your model, whatever it is, into a system of equations, let the computer solve them over a given period, and, voila, you have a simulation.

In this new world of computer modelling, an oft-quoted remark made in the 1970s by the statistician George Box remains a useful rule of thumb: ‘all models are wrong, but some are useful’. He meant, of course, that while the new simulations should never be mistaken for the real thing, their features might yet inform us about aspects of reality that matter....MUCH MORE
We have so many posts on models and modelling that it is probably easiest to do a Google search of Climateer Investing: model

Some of the more interesting hits on the first couple pages (I'm not kidding we have hundreds of posts):

"Embracing Wynne Godley, an Economist Who Modeled the Crisis"
Insurance: "CEO FORUM: Gen Re's Tad Montross on model dependency" (BRK.A)
Why Economic Models Are Always Wrong
Quantology: Coupled Hidden Markov Models--USD/CHF and Gold, for example
"Economists' Grail: A Post-Crash Model"
Incorporating the Rentier Sectors into a Financial Model
NY Fed Treasury Spread Model: Probability Of Double-Dip Recession Is Zero
Book Review: "Models.Behaving.Badly: Why Confusing Illusion with Reality Can Lead to Disaster, on Wall Street and in Life "
The continuing relevance of the bootlegger-and-Baptist model
"First Mathematical Model of Cow Behaviour "
Like Phillips, Irving Fisher Was a Plumber: Hydraulic Models of the Economy
Climate prediction: No model for success
Ya Put Your Financial Model on Top of Your Climate Model; Borrow Money to Buy Derivatives...
Modelling vs. Science
The Financial Modelers' Manifesto
After the Crash: How Software Models Doomed the Markets
Airspace Closure Was Exacerbated by Too Much Modeling, Too Little Research
UPDATED: The Bogus Hurricane Models that Cost Florida Billions
Re-thinking Risk Management: Why the Mindset Matters More Than the Model
Insurance: Is the industry too reliant on models? (BRK.B)

And many, many more.