Well, maybe a little, but it doesn’t help.
From time to time we at Ventana encounter consulting engagements where the problem space is already occupied by other models. Typically, these are big, detailed models from academic or national lab teams who’ve been working on them for a long time. For example, in an aerospace project we ran into detailed point-to-point trip generation models and airspace management simulations with every known airport and aircraft in them. They were good, but cumbersome and expensive to run. Our job was to take a top-down look at the big picture, integrating the knowledge from the big but narrow models. At first there was a lot of resistance to our intrusion, because we consumed some of the budget, until it became evident that the existence of the top-down model added value to the bottom-up models by placing them in context, making their results more relevant. The benefit was mutual, because the bottom-up models provided grounding for our model that otherwise would have been very difficult to establish. I can’t quite say that we became one big happy family, but we certainly developed a productive working relationship.
I think situations involving complementary models are more common than head-to-head competition among models that serve the same purpose. Even where head-to-head competition does exist, it’s healthy to have multiple models, especially if they embody different methods. (The trouble with global climate policy is that we have many models that mostly embody the same general equilibrium assumptions, and thus differ only in detail.) Rather than getting into methodological pissing matches, modelers should be seeking the synergy among their efforts and making it known to decision makers. That helps to grow the pie for all modeling efforts, and produces better decisions.
Certainly there are exceptions. I once ran across a competing vendor doing marketing science for a big consumer products company. We were baffled by the high R^2 values they were reporting (.92 to .98), so we reverse engineered their model from the data and some slides (easy, because it was a linear regression). It turned out that the great fits were due to the use of 52 independent parameters to capture seasonal variation on a weekly basis. Since there were only 3 years of data (i.e. 3 points per parameter), we dubbed that the “variance eraser.” Replacing the 52 parameters with a few targeted at holidays and broad variations resulted in more realistic fits, and also revealed problems with inverted signs (presumably due to collinearity) and other typical pathologies. That model deserved to be displaced. Still, we learned something from it: when we looked cross-sectionally at several variants for different products, we discovered that coefficients describing the sales response to advertising were dependent on the scale of the product line, consistent with our prior assertion that effects of marketing and other activities were multiplicative, not additive.
The reality is that the need for models is almost unlimited. The physical sciences are fairly well formalized, but models span a discouragingly small fraction of the scope of human behavior and institutions. We need to get the cost of providing insight down, not restrict the supply through infighting. The real enemy is seldom other models, but rather superstition, guesswork and propaganda.
I have a different take on “can we all just get along?” Rather than sitting around a campfire munching on smores and singing Kumbaya, we need Marquis of Queensbury Rules. When modellers compete, modellers learn from each other.
> The physical sciences are fairly well formalized, but models span a discouragingly small fraction of the scope of human behavior and institutions.
< Models in physical sciences emerge very slowly, require a huge amount of observation, conjecture, reflection and testing before being accepted hypotheses, and are useful in strictly circumscribed conditions. At any time, there are competing models in physical sciences — even competing conceptual frameworks — to explain observed phenomena. My experience in system dynamics is pretty much the same. For the most part, we play in social science fields with a conceptual framework that differs from a more established conceptual framework. We offer a different take on things, a competing explanation. At best we can ask questions that another conceptual framework could not even generate, and our explanation is useful.
JT
I had to look up Queensbury rules. Building models that way is exhausting.
Whether we need Kumbaya or an honorable fight, I think part of the challenge is getting the users of models to entertain and support multiple approaches. That’s tough, especially in economic, energy and climate policy, where the system is almost hopelessly locked into equilibrium assumptions. As a result, the models are far down their learning curves, and expensive to supplement with similar disequilibrium or agent models. We desperately need the alternatives though.