Distilling Free-Form Natural Laws from Experimental Data

An interesting paper of that name came out in Science two years ago. There’s a neat video:

For centuries, scientists have attempted to identify and document analytical laws that underlie physical phenomena in nature. Despite the prevalence of computing power, the process of finding natural laws and their corresponding equations has resisted automation. A key challenge to finding analytic relations automatically is defining algorithmically what makes a correlation in observed data important and insightful. We propose a principle for the identification of nontriviality. We demonstrated this approach by automatically searching motion-tracking data captured from various physical systems, ranging from simple harmonic oscillators to chaotic double-pendula. Without any prior knowledge about physics, kinematics, or geometry, the algorithm discovered Hamiltonians, Lagrangians, and other laws of geometric and momentum conservation. The discovery rate accelerated as laws found for simpler systems were used to bootstrap explanations for more complex systems, gradually uncovering the “alphabet” used to describe those systems.

The Eureqa application used to mine data for relationships has been released at the authors’ Cornell site.

I think an interesting question is, will this approach work on noisy or ill-defined systems like climate or organizations? My guess is that it will have the same limitations as human-produced science. There’s a reason that a lot of physical laws were nailed down centuries ago, but our models of biological, economic and social phenomena are still pretty limited.

Modeling is not optional


The design of a complex regulator often includes the making of a model of the system to be regulated. The making of such a model has hitherto been regarded as optional, as merely one of many possible ways.

In this paper a theorem is presented which shows, under very broad conditions, that any regulator that is maximally both successful and simple must be isomorphic with the system being regulated.  (The exact assumptions are given.) Making a model is thus necessary.

The theorem has the interesting corollary that the living brain, so far as it is to be successful and efficient as a regulator for survival, must proceed, in learning, by the formation of a model (or models) of its environment.

That’s from a classic cybernetics paper by Conant & Ashby (Int. J. Systems Sci., 1970, vol. 1, No. 2, 89-97). It even has an interesting web project dedicated to it.

It’s one of several on a nice reading list on the foundations of complexity that I ran across at the Sante Fe Institute. Some of the pdfs are here.

Drunker than intended and overinvested

Erling Moxnes on the dangers of forecasting without structural insight and the generic structure behind getting too drunk and underestimating delays when investing in a market, with the common outcome of  instability.

More on drinking dynamics here, implemented as a game on Forio (haven’t tried it yet – curious about your experience if you do).

The seven-track melee

In boiled frogs I explored the implications of using local weather to reason about global climate. The statistical fallacies (local = global and weather = climate) are one example of the kinds of failures on my list of reasons for science denial.

As I pondered the challenge of upgrading mental models to cope with big problems like climate, I ran across a great paper by Barry Richmond (creator of STELLA, and my first SD teacher long ago). He inventories seven systems thinking skills, which nicely dovetail with my thinking about coping with complex problems.

Some excerpts:

Skill 1: dynamic thinking

Dynamic thinking is the ability to see and deduce behavior patterns rather than focusing on, and seeking to predict, events. It’s thinking about phenomena as resulting from ongoing circular processes unfolding through time rather than as belonging to a set of factors. …

Skill 2: closed-loop thinking

The second type of thinking process, closed-loop thinking, is closely linked to the first, dynamic thinking. As already noted, when people think in terms of closed loops, they see the world as a set of ongoing, interdependent processes rather than as a laundry list of one-way relations between a group of factors and a phenomenon that these factors are causing. But there is more. When exercising closed-loop thinking, people will look to the loops themselves (i.e., the circular cause-effect relations) as being responsible for generating the behavior patterns exhibited by a system. …

Skill 3: generic thinking

Just as most people are captivated by events, they are generally locked into thinking in terms of specifics. … was it Hitler, Napoleon, Joan of Arc, Martin Luther King who determined changes in history, or tides in history that swept these figures along on their crests? … Apprehending the similarities in the underlying feedback-loop relations that generate a predator-prey cycle, a manic-depressive swing, the oscillation in an L-C circuit, and a business cycle can demonstrate how generic thinking can be applied to virtually any arena.

Skill 4: structural thinking

Structural thinking is one of the most disciplined of the systems thinking tracks. It’s here that people must think in terms of units of measure, or dimensions. Physical conservation laws are rigorously adhered to in this domain. The distinction between a stock and a flow is emphasized. …

Skill 5: operational thinking

Operational thinking goes hand in hand with structural thinking. Thinking operationally means thinking in terms of how things really work—not how they theoretically work, or how one might fashion a bit of algebra capable of generating realistic-looking output. …

Skill 6: continuum thinking

Continuum thinking is nourished primarily by working with simulation models that have been built using a continuous, as opposed to discrete, modeling approach. … Although, from a mechanical standpoint, the differences between the continuous and discrete formulations may seem unimportant, the associated implications for thinking are quite profound. An “if, then, else” view of the world tends to lead to “us versus them” and “is versus is not” distinctions. Such distinctions, in turn, tend to result in polarized thinking.

Skill 7: scientific thinking

… Let me begin by saying what scientific thinking is not. My definition of scientific thinking has virtually nothing to do with absolute numerical measurement. … To me, scientific thinking has more to do with quantification than measurement. … Thinking scientifically also means being rigorous about testing hypotheses. … People thinking scientifically modify only one thing at a time and hold all else constant. They also test their models from steady state, using idealized inputs to call forth “natural frequency responses.”

When one becomes aware that good systems thinking involves working on at least these seven tracks simultaneously, it becomes a lot easier to understand why people trying to learn this framework often go on overload. When these tracks are explicitly organized, and separate attention is paid to develop each skill, the resulting bite-sized pieces make the fare much more digestible. …

The connections among the various physical, social, and ecological subsystems that make up our reality are tightening. There is indeed less and less “away,” both spatially and temporally, to throw things into. Unfortunately, the evolution of our thinking capabilities has not kept pace with this growing level of interdependence. The consequence is that the problems we now face are stubbornly resistant to our interventions. To “get back into the foot race,” we will need to coherently evolve our educational system

… By viewing systems thinking within the broader context of critical thinking skills, and by recognizing the multidimensional nature of the thinking skills involved in systems thinking, we can greatly reduce the time it takes for people to apprehend this framework. As this framework increasingly becomes the context within which we think, we will gain much greater leverage in addressing the pressing issues that await us …

Source: Barry Richmond, “Systems thinking: critical thinking skills for the 1990s and beyond” System Dynamics Review Volume 9 Number 2 Summer 1993

That was 18 years ago, and I’d argue that we’re still not back in the race. Maybe recognizing the inherent complexity of the challenge and breaking it down into digestible chunks will help though.

Boiled frogs, pattern recognition and climate policy

A friend (who shall remain nameless to avoid persecution) relates:

I just have to share a somewhat funny, mostly frustrating story.  XXX and I went to Montreal to see U2 last night …

So at the end of the show, it started raining.  Not a light rain but a sharp, arctic windy rain to the point we were all saturated within minutes.  No exaggeration.  It took us 2 hours to get out of the stadium, to the Metro, and back to the hotel because there were 80,000 people trying to get out.  It was purgatory.  It was after 3 am before we finally got to sleep.

OK, that’s not the story either.  My point, besides sharing the night with you, is to explain my state of mind leading up to the customs experience. At the border, things were going smoothly … until the security guard asked us what we do for work.  When I told him what I do, he retorted, “Global warming is not happening. Have you seen the cold weather we’ve had for the past five years?” Of course, I couldn’t let that just sit there.  I explained climate change is not just warmer weather but more extreme events and  weather is different from climate.  So then we actually started arguing about this and I got all fired up …

Now the frustrating part.  As we drove away, though, I just felt so despondent that this is the battle with much of the public that we face.  How can we convince people to act and to demand their government to act when they really don’t think there is a problem?  I’ve been thinking we should utilize some of our time to reach out to the masses who are not looking to improve their mental models of climate change because they don’t even think such a thing exists.   …

The fundamental problem, I think, is that evidence for climate change relies on a variety of measurements dispersed over the globe, and physics and models at a variety of levels of complexity. Yet neither of these is accessible to ordinary people. Therefore, when presented with a mix of inconvenience of reducing emissions, false balance, and pseudoscience, people default to a “trust no one” approach, only believing what they can perceive with their own senses: local weather.

That’s a problem, for several reasons. First, local weather is extremely noisy, and only loosely correlated with global energy balance. So, even a person with unbiased weather perception and perfect recall over a long time period really shouldn’t be drawing any conclusions about global conditions from local measurements.

Second, no one has unbiased perception and perfect recall over long periods. Consider the “cold weather we’ve had for the past five years.” Here’s the temperature from the USHCN Enosburg Falls, VT station, near the border crossing, from 1891 through the end of 2009, in degrees Fahrenheit:

It’s hard to imagine something happening in 2010 and 2011 that could lead one to feel good about conclusively calling a 5-yr cold spell. In fact, almost the only thing you can see here is a slight upward trend.

Continue reading “Boiled frogs, pattern recognition and climate policy”

The Economic Long Wave

This is John Sterman’s model of long waves (long-duration economic cycles), driven by capital accumulation dynamics. This version is replicated from a JEBO article,

STERMAN, J. D. (1985) A Behavioral Model of the Economic Long Wave. Journal of Economic Behavior and Organization, 6, 17-53.

There’s some interesting related literature (including other economic models in this library). From Sterman’s publications list:

STERMAN, J. D. & MOSEKILDE, E. (1994) Business Cycles and Long Waves: A Behavioral, Disequilibrium Perspective. IN SEMMLER, W. (Ed.) Business Cycles: Theory and Empirical Methods. Boston, Kluwer Academic Publishers.

STERMAN, J. D. (1994) The Economic Long Wave: Theory and Evidence. IN SHIMADA, T. (Ed.) An Introduction to System Dynamics. Tokyo.

STERMAN, J. D. (2002) A Behavioral Model of the Economic Long Wave. IN EARL, P. E. (Ed.) The Legacy of Herbert Simon in Economic Analysis. Cheltenham, UK, Edward Elgar.

STERMAN, J. D. (1985) An Integrated Theory of the Economic Long Wave. Futures, 17, 104-131.

RASMUSSEN, S., MOSEKILDE, E. & STERMAN, J. D. (1985) Bifurcations and Chaotic Behavior in a Simple Model of the Economic Long Wave. System Dynamics Review, 1, 92-110.

STERMAN, J. D. (1983) The Long Wave. Science, 219, 1276.

KAMPMANN, C., HAXHOLDT, C., MOSEKILDE, E. & STERMAN, J. D. (1994) Entrainment in a Disaggregated Economic Long Wave Model. IN LEYDESDORFF, L. & VAN DEN BESSELAAR, P. (Eds.) Evolutionary Economics and Chaos Theory. London, Pinter.

MOSEKILDE, E., LARSEN, E. R., STERMAN, J. D. & THOMSEN, J. S. (1993) Mode Locking and Nonlinear Entrainment of Macroeconomic Cycles. IN DAY, R. & CHEN, P. (Eds.) Nonlinear Economics and Evolutionary Economics. New York, Oxford University Press.

MOSEKILDE, E., THOMSEN, J. S. & STERMAN, J. D. (1992) Nonlinear Interactions in the Economy. IN HAAG, G., MÜLLER, U. & TROITZSCH, K. (Eds.) Economic Evolution and Demographic Change. Berlin, Springer Verlag.

THOMSEN, J. S., MOSEKILDE, E. & STERMAN, J. D. (1991) Hyperchaotic Phenomena in Dynamic Decision Making. IN SINGH, M. G. & TRAVÉ-MASSUYÈS, L. (Eds.) Decision Support Systems and Qualitative Reasoning. Amsterdam, Elsevier Science Publishers.

THOMSEN, J. S., MOSEKILDE, E., LARSEN, E. R. & STERMAN, J. D. (1991) Mode-Locking and Chaos in a Periodically Driven Model of the Economic Long Wave. IN EBELING, W. (Ed.) Models of Self Organization in Complex Systems. Berlin, Akademie Verlag.

STERMAN, J. D. (1988) Nonlinear Dynamics in the World Economy: The Economic Long Wave. IN CHRISTIANSEN, P. & PARMENTIER, R. (Eds.) Structure, Coherence, and Chaos in Dynamical Systems. Manchester, Manchester University Press.

STERMAN, J. D. (1987) Debt, Default, and Long Waves: Is History Relevant? IN BOECKH, A. (Ed.) The Escalation in Debt and Disinflation: Prelude to Financial Mania and Crash? Montreal, BCA Publications.

STERMAN, J. D. (1987) An Integrated Theory of the Economic Long Wave. IN WANG, Q., SENGE, P., RICHARDSON, G. P. & MEADOWS, D. H. (Eds.) Theory and Application of System Dynamics. Beijing, New Times Press.

STERMAN, J. D. (1987) The Economic Long Wave: Theory and Evidence. IN VASKO, T. (Ed.) The Long Wave Debate. Berlin, Springer Verlag.

RASMUSSEN, S., MOSEKILDE, E. & STERMAN, J. D. (1987) Bifurcations and Chaotic Behavior in a Simple Model of the Economic Long Wave. IN WANG, Q., SENGE, P., RICHARDSON, G. P. & MEADOWS, D. H. (Eds.) Theory and Application of System Dynamics. Beijing, New Times Press.

And from Christian Kampmann,

“The Role of Prices in Long Wave Entrainment” (with C. Haxholdt, E. Mosekilde, and J.D. Sterman), International System Dynamics Conference, Stirling, U.K. and at the Spring 1994 ORSA/TIMS conference, Boston, MA. 1994.
“Disaggregating a simple model of the economic long wave” International Conference of the System Dynamics Society, Keystone, CO, 1985.
The long wave model was the guine pig for Kampmann’s interesting ’96 conference paper that combined a graph-theoretic identification of a set of feedback loops having independent gains with eigenvalue analysis,
Kampmann, Christian E.   Feedback Loop Gains and System Behavior
There also used to be a nifty long wave game, programmed on NEC minicomputers (32k memory?), but I’ve lost track of it. I’d be interested to here of a working version.

Economic Cycles: Underlying Causes

Nathaniel Mass’ model of economic cycles, replicated from his 1975 book, Economic Cycles: An Analysis of Underlying Causes, which unfortunately seems to have disappeared from the Productivity Press site (though you can still find used copies).

I haven’t checked, but I’m guessing that the model is quite similar to that in his PhD thesis, which you can get from MIT libraries here. Here’s the abstract:

The models: mass2.mdl mass2.vpm

These don’t have units defined, unfortunately – I’d love to have a copy with units if you’re so inclined.

The Dynamics of Commodity Production Cycles

These classic models are from Dennis Meadows’ dissertation, the Dynamics of Commodity Production Cycles:

While times have changed, the dynamics described by these models are still widespread.

These versions should work in all recent Vensim versions:

DLMhogs2.vpm DLMhogs2.mdl

DLMgeneric2.vpm DLMgeneric2.mdl