Author: Tom

Stimulus regret revisited

A year ago I wrote,

Stimulus regret seems to be pretty widespread now. The undercurrent seems to be that, because unemployment is still 10% etc., the stimulus didn’t work …. This conclusion is based on pattern matching thinking. Pattern matching assumes simple A->B correlation: Stimulus->Unemployment. Working backwards from that assumption, one concludes from ongoing high unemployment and the fact that stimulus did occur that the correlation between stimulus and unemployment is low.

There are two problems with this logic. First, there are many confounding factors in the A->B relationship that could be responsible for ongoing problems. Second, there’s feedback between A and B, which also means that there are (possibly large) intervening stocks (integrations, accumulations). Stocks decouple the temporal relationship between A and B, so that pattern matching doesn’t work.

Today, Paul Krugman decries similar thinking, and identifies a third misperception (that an effect may be small either because of weak causal links, or because the cause was small),

It’s kind of annoying when people claim that I said the stimulus would work; how much noisier could I have been in warning both that it was grossly inadequate, and that by claiming that a far-too-small stimulus was just right, Obama would discredit the whole idea?

Krugman points out that evaluating suites of predictions, not just a single outcome, provides a way to discriminate between competing mental models:

Of course, the WSJ also said that the stimulus wouldn’t work. The difference was in how it was supposed to fail.

The WSJ view was that federal borrowing would crowd out private spending by driving interest rates sky-high, that the bond vigilantes would destroy the economy. …

My view was that government borrowing in a liquidity trap does not drive up rates, and indeed that rates would stay low as long as the economy stayed depressed.

How it turned out.

That’s a pretty clear test; among other things, you would have lost a lot of money if you believed the WSJ view.

The problem remains that there is relatively little of such thoughtful evaluation going on in the public discourse.

For a politician evaluated by people who ignore system structure, this is a no-win situation. As long as things get worse, blame follows, regardless of what policy is chosen.

The rise of systems sciences

The Google books ngram viewer nicely documents the rise of various systems science disciplines, from about the time of Maxwell’s landmark 1868 paper, On Governors:

Click to enlarge.

We still have a long way to go though:

Further reading:

Limits to bathtubs

Danger lurks in the bathtub – not just slips, falls, and Norman Bates, but also bad model formulations.

A while ago, after working with my kids to collect data on our bathtub, I wrote My bathtub is nonlinear.

We grabbed a sheet of graph paper, fat pens, a yardstick, and a stopwatch and headed for the bathtub. …

When the tub was full, we made a few guesses about how long it might take to empty, then started the clock and opened the drain. Every ten or twenty seconds, we’d stop the timer, take a depth reading, and plot the result on our graph. …

To my astonishment, the resulting plot showed a perfectly linear decline in water depth, all the way to zero (as best we could measure). In hindsight, it’s not all that strange, because the tub tapers at the bottom, so that a constant linear decline in the outflow rate corresponds with the declining volumetric flow rate you’d expect (from decreasing pressure at the outlet as the water gets shallower). Still, I find it rather amazing that the shape of the tub (and perhaps nonlinearity in the drain’s behavior) results in such a perfectly linear trajectory.

It turns out that my attribution of the linear time vs. depth profile was sloppy – the behavior has a little to do with tub shape, and a lot to do with nonlinearity in the draining behavior. In a nice brief from the SD conference, Pål Davidsen, Erling Moxnes, Mauricio Munera Sánchez and David Wheat explain why:

… in the 16th century the Italian scientist Evangelista Torricelli found the relationship between water height and outflow to be nonlinear.

… Torricelli may have reasoned as follows. Let a droplet of water fall frictionless outside the tank from the same height … as the surface of the water. Gravitation will make the droplet accelerate. As the droplet passes the bottom of the tank, its kinetic energy will equal the loss of potential energy … Reorganizing this equation Torricelli found the following nonlinear expression for speed as a function of height

v = SQRT(2*g*h)

Then Torricelli moved inside the tank and reasoned that the same must apply there. …

Assuming that the water tank is a cylinder with straight walls … The outflow is given by the square root of volume; it is not a linear function of volume.

– “A note on the bathtub analogy,” ISDC 2011; final proceedings aren’t online yet but presumably will be here eventually.

In hindsight, this ought to have been obvious to me, because bathtubs clearly don’t exhibit the heavy-right-tail behavior of a first order linear draining process. The difference matters:

The bathtub analogy has been used extensively to illustrate stock and flow relationships. Because this analogy is frequently used, System Dynamicists should be aware that the natural outflow of water from a bathtub is a nonlinear function of water volume. A questionnaire suggests that students with one year or more of System Dynamics training tend to assume a linear relationship when asked to model a water outflow driven by gravity. We present Torricelli’s law for the outflow and investigate the error caused by assuming linearity. We also construct an “inverted funnel” which does behave like a linear system. We conclude by pointing out that the nonlinearity is of no importance for the usefulness of bathtubs or funnels as analogies. On the other hand, simplified analogies could make modellers overconfident in linear formulations and not able to address critical remarks from physicists or other specialists.

I’ve been doing SD for over two decades, and have the physical science background to know better, but found it a little too easy to assume a linear bathtub as a mental model, without inquiring very deeply when confronted with disconfirming data. For me, this is a nice cautionary lesson, that we forget the roots of system dynamics in engineering at our own peril.

My implementation of the model is in my library.

Is London a big whale?

Why do cities survive atom bombs, while companies routinely go belly up?

Geoffrey West on The Surprising Math of Cities and Corporations:

There’s another interesting video with West in the conversations at Edge.

West looks at the metabolism of cities, and observes scale-free behavior of good stuff (income, innovation, input efficiency) as well as bad stuff (crime, disease – products of entropy). The destiny of cities, like companies, is collapse, except to the extent that they can innovate at an accelerating rate. Better hope the Singularity is on schedule.

Thanks to whoever it was at the SD conference who pointed this out!

Distilling Free-Form Natural Laws from Experimental Data

An interesting paper of that name came out in Science two years ago. There’s a neat video:

For centuries, scientists have attempted to identify and document analytical laws that underlie physical phenomena in nature. Despite the prevalence of computing power, the process of finding natural laws and their corresponding equations has resisted automation. A key challenge to finding analytic relations automatically is defining algorithmically what makes a correlation in observed data important and insightful. We propose a principle for the identification of nontriviality. We demonstrated this approach by automatically searching motion-tracking data captured from various physical systems, ranging from simple harmonic oscillators to chaotic double-pendula. Without any prior knowledge about physics, kinematics, or geometry, the algorithm discovered Hamiltonians, Lagrangians, and other laws of geometric and momentum conservation. The discovery rate accelerated as laws found for simpler systems were used to bootstrap explanations for more complex systems, gradually uncovering the “alphabet” used to describe those systems.

The Eureqa application used to mine data for relationships has been released at the authors’ Cornell site.

I think an interesting question is, will this approach work on noisy or ill-defined systems like climate or organizations? My guess is that it will have the same limitations as human-produced science. There’s a reason that a lot of physical laws were nailed down centuries ago, but our models of biological, economic and social phenomena are still pretty limited.

Modeling is not optional

EVERY GOOD REGULATOR OF A SYSTEM MUST BE A MODEL OF THAT SYSTEM

The design of a complex regulator often includes the making of a model of the system to be regulated. The making of such a model has hitherto been regarded as optional, as merely one of many possible ways.

In this paper a theorem is presented which shows, under very broad conditions, that any regulator that is maximally both successful and simple must be isomorphic with the system being regulated.  (The exact assumptions are given.) Making a model is thus necessary.

The theorem has the interesting corollary that the living brain, so far as it is to be successful and efficient as a regulator for survival, must proceed, in learning, by the formation of a model (or models) of its environment.

That’s from a classic cybernetics paper by Conant & Ashby (Int. J. Systems Sci., 1970, vol. 1, No. 2, 89-97). It even has an interesting web project dedicated to it.

It’s one of several on a nice reading list on the foundations of complexity that I ran across at the Sante Fe Institute. Some of the pdfs are here.

Drunker than intended and overinvested

Erling Moxnes on the dangers of forecasting without structural insight and the generic structure behind getting too drunk and underestimating delays when investing in a market, with the common outcome of  instability.

More on drinking dynamics here, implemented as a game on Forio (haven’t tried it yet – curious about your experience if you do).

The seven-track melee

In boiled frogs I explored the implications of using local weather to reason about global climate. The statistical fallacies (local = global and weather = climate) are one example of the kinds of failures on my list of reasons for science denial.

As I pondered the challenge of upgrading mental models to cope with big problems like climate, I ran across a great paper by Barry Richmond (creator of STELLA, and my first SD teacher long ago). He inventories seven systems thinking skills, which nicely dovetail with my thinking about coping with complex problems.

Some excerpts:

Skill 1: dynamic thinking

Dynamic thinking is the ability to see and deduce behavior patterns rather than focusing on, and seeking to predict, events. It’s thinking about phenomena as resulting from ongoing circular processes unfolding through time rather than as belonging to a set of factors. …

Skill 2: closed-loop thinking

The second type of thinking process, closed-loop thinking, is closely linked to the first, dynamic thinking. As already noted, when people think in terms of closed loops, they see the world as a set of ongoing, interdependent processes rather than as a laundry list of one-way relations between a group of factors and a phenomenon that these factors are causing. But there is more. When exercising closed-loop thinking, people will look to the loops themselves (i.e., the circular cause-effect relations) as being responsible for generating the behavior patterns exhibited by a system. …

Skill 3: generic thinking

Just as most people are captivated by events, they are generally locked into thinking in terms of specifics. … was it Hitler, Napoleon, Joan of Arc, Martin Luther King who determined changes in history, or tides in history that swept these figures along on their crests? … Apprehending the similarities in the underlying feedback-loop relations that generate a predator-prey cycle, a manic-depressive swing, the oscillation in an L-C circuit, and a business cycle can demonstrate how generic thinking can be applied to virtually any arena.

Skill 4: structural thinking

Structural thinking is one of the most disciplined of the systems thinking tracks. It’s here that people must think in terms of units of measure, or dimensions. Physical conservation laws are rigorously adhered to in this domain. The distinction between a stock and a flow is emphasized. …

Skill 5: operational thinking

Operational thinking goes hand in hand with structural thinking. Thinking operationally means thinking in terms of how things really work—not how they theoretically work, or how one might fashion a bit of algebra capable of generating realistic-looking output. …

Skill 6: continuum thinking

Continuum thinking is nourished primarily by working with simulation models that have been built using a continuous, as opposed to discrete, modeling approach. … Although, from a mechanical standpoint, the differences between the continuous and discrete formulations may seem unimportant, the associated implications for thinking are quite profound. An “if, then, else” view of the world tends to lead to “us versus them” and “is versus is not” distinctions. Such distinctions, in turn, tend to result in polarized thinking.

Skill 7: scientific thinking

… Let me begin by saying what scientific thinking is not. My definition of scientific thinking has virtually nothing to do with absolute numerical measurement. … To me, scientific thinking has more to do with quantification than measurement. … Thinking scientifically also means being rigorous about testing hypotheses. … People thinking scientifically modify only one thing at a time and hold all else constant. They also test their models from steady state, using idealized inputs to call forth “natural frequency responses.”

When one becomes aware that good systems thinking involves working on at least these seven tracks simultaneously, it becomes a lot easier to understand why people trying to learn this framework often go on overload. When these tracks are explicitly organized, and separate attention is paid to develop each skill, the resulting bite-sized pieces make the fare much more digestible. …

The connections among the various physical, social, and ecological subsystems that make up our reality are tightening. There is indeed less and less “away,” both spatially and temporally, to throw things into. Unfortunately, the evolution of our thinking capabilities has not kept pace with this growing level of interdependence. The consequence is that the problems we now face are stubbornly resistant to our interventions. To “get back into the foot race,” we will need to coherently evolve our educational system

… By viewing systems thinking within the broader context of critical thinking skills, and by recognizing the multidimensional nature of the thinking skills involved in systems thinking, we can greatly reduce the time it takes for people to apprehend this framework. As this framework increasingly becomes the context within which we think, we will gain much greater leverage in addressing the pressing issues that await us …

Source: Barry Richmond, “Systems thinking: critical thinking skills for the 1990s and beyond” System Dynamics Review Volume 9 Number 2 Summer 1993

That was 18 years ago, and I’d argue that we’re still not back in the race. Maybe recognizing the inherent complexity of the challenge and breaking it down into digestible chunks will help though.