A while back I made a video about spreadsheets, that makes some points about open-loop models vs. real, closed-loop dynamic models:
The short version is that people tend to build this:
when reality works like this:
I think there are some understandable reasons to prefer the first, simpler view:
- Just understanding the dynamics of accumulation (here, the vehicle stock) may be mind-blowing enough without adding feedback complexity.
- It’s a start, and certainly better than no model or extrapolation!
Some of the reasons these models get built are a little less appetizing:
- In the short run, some loops really aren’t closed (though the short run is rarely as short as you think, and myopia gets you in trouble).
- They’re quicker and cheaper to build (if you don’t mind less insight).
- Dynamic modeling skills, both for construction and consumption, are not very widespread.
- Open-loop models are easier to calibrate (but a calibration neglecting accumulation and feedback is likely bogus and misleading).
- Open-loop models are easier to manipulate to produce a desired outcome.
I think the last point is key. At Ventana, we’ve discussed – only partly in jest – creating a “propaganda mode” for Vensim and Ventity. This would automate the discovery of a parameterization of a model that both fits history and makes a preferred policy optimal.
Perhaps the ultimate example of this is the RMSM model. 20 years ago, this was the World Bank’s preferred tool for country modeling. When Gerald Barney and Weishuang Qu replicated the model in Vensim, they discovered that is was full of disconnected trees of causality. That would permit creation of a scenario in which GDP growth marched along merrily without any water, for example. Politically, this was actually a feature, not a bug, because some users simply didn’t want to know that their pet project would displace a lot of people or destroy resources.
I think the solution here is to equip people to ask the right questions that close loops. Once there’s an appetite for dynamic, operational thinking, we can supply good modelers to provide the tools.
Excellent points, Tom! Too often speed is the main driver for policy decisions (whether in politics, economics or the business world in general). This intentially puts aside rigorous thinking and debating about the possible effects, root-causes of present status, and essential feedback-loops.
“Simple” models (also conversations between the players involved) including the above are a valuable start, worth pursuing and driving forward.
Hi Tom,
“Propaganda Mode” is a great phrase and, unfortunately descriptive of what a lot of folks prefer.
As you might imagine, over decades working in or near the public sector, I’ve gotten many requests for just that sort of work.
Very early on (actually my first consulting work after formally setting up shop), I learned how dangerous my own assumptions could be both analytically and in over-promising a result to a prospective client, who indeed wanted a particular result to use politically. It was a Medicaid project, a topic I actually knew a lot about. Not as much as I thought though. I was so over-confident that it never occurred to me that the client and I might be wrong. The project was purely a statistical analysis of an effect in 57 NYS counties as well as the aggregate. Painfully, the actual result differed from the expected in every single case. Yes, every case.
I went to my next meeting with client staff with a blank check prepared to buy back the advance they’d already paid. That added an extra layer of pain to the whole process, especially so for a start-up.
But it turned out to be fortunate that that lesson was learned so early.
Our next project was a SD modeling project and when approached by another organization to “prove the Governor was wrong,” I told the prospect:
1. If you want a guaranteed result, you have to pay me enough to retire on. (Nobody’s ever taken me up on that. Sigh.)
2. While we may dicker over wording of a report, if you want my name on it, I get final editorial control. Take my name off and you can do whatever you want. (That’s happened, but I never worked with them again. And because she was touting results before we began the analysis, I even fired one client.)
3. You, the client get final control over distribution.
4. Sometimes, if the client is going to be deeply, actively engaged in the model building process, I’ll informally promise they’ll believe an unwanted result. I did in this particular case. The Governor was right, the client was grumpy, but because he was very much part of the model building process, he came to understand why the Governor was right and he was wrong. Now, years later, the only other person to see the model is my wife. (Well, there’s one funny story with a modest exception to that, but it’ll take a couple beers to hear it.)
Will mull on the notion of enabling an actual “propaganda mode” in Vensim and Ventity. Perhaps presumptuous of me to offer gratuitous advice on this, but I’d be disinclined to enable it. Your product would then begin to look more like software for influence diagrams or decision trees, confusing your clients, prospects, and their audiences, and muddying your focus and strengths. Of course, I haven’t modeled this so …
One possible exception might be as a teaching and client self-auditing tool. It might be a design and coding nightmare, but it might be interesting to be able to cycle between feedback and propaganda modes to see the differences.
In any case, consider the phrase “propaganda mode” to be stolen!