Author: Tom

Dynamic simulation the hard way

When Alan Turing was born 100 years ago, on June 23, 1912, a computer was not a thing—it was a person. Computers, most of whom were women, were hired to perform repetitive calculations for hours on end. The practice dated back to the 1750s, when Alexis-Claude ­Clairaut recruited two fellow astronomers to help him plot the orbit of Halley’s comet. ­Clairaut’s approach was to slice time into segments and, using Newton’s laws, calculate the changes to the comet’s position as it passed Jupiter and Saturn. The team worked for five months, repeating the process again and again as they slowly plotted the course of the celestial bodies.

Today we call this process dynamic simulation; Clairaut’s contemporaries called it an abomination. They desired a science of fundamental laws and beautiful equations, not tables and tables of numbers. Still, his team made a close prediction of the perihelion of Halley’s comet. Over the following century and a half, computational methods came to dominate astronomy and engineering.

From Turing’s Enduring Importance in Technology Review.

Calibrate your confidence bounds: an updated Capen Quiz

Forecasters are notoriously overconfident. This applies to nearly everyone who predicts anything, not just stock analysts. A few fields, like meteorology, have gotten a handle on the uncertainty in their forecasts, but this remains the exception rather than the rule.

Having no good quantitative idea of uncertainty, there is an almost universal tendency for people to understate it. Thus, they overestimate the precision of their own knowledge and contribute to decisions that later become subject to unwelcome surprises.

A solution to this problem involves some better understanding of how to treat uncertainties and a realization that our desire for preciseness in such an unpredictable world may be leading us astray.

E.C. Capen illustrated the problem in 1976 with a quiz that asks takers to state 90% confidence intervals for a variety of things – the length of the Golden Gate bridge, the number of cars in California, etc. A winning score is 9 out of 10 right. 10 out of 10 indicates that the taker was underconfident, choosing ranges that are too wide.

Ventana colleague Bill Arthur has been giving the quiz to clients for years. In fact, it turns out that the vast majority of takers are overconfident in their knowledge – they choose ranges that are too narrow, and get only a three or four questions right. CEOs are the worst – if you score zero out of 10, you’re c-suite material.

My kids and I took the test last year. Using what we learned, we expanded the variance on our guesses of the weight of a giant pumpkin at the local coop – and as a result, brought the monster home.

Now that I’ve taken the test a few times, it spoils the fun, so last time I was in a room for the event, I doodled an updated quiz. Here’s your chance to calibrate your confidence intervals:

For each question, specify a range (minimum and maximum value) within which you are 80% certain that the true answer lies. In other words, in an ideal set of responses, 8 out of 10 answers will contain the truth within your range.

Example*:

The question is, “what was the winning time in the first Tour de France bicycle race, in 1903?”

Your answer is, “between 1 hour and 1 day.”

Your answer is wrong, because the truth (94 hours, 33 minutes, 14 seconds) does not lie within your range.

Note that it doesn’t help to know a lot about the subject matter – precise knowledge merely requires you to narrow your intervals in order to be correct 80% of the time.

Now the questions:

  1. What is the wingspan of an Airbus A380-800 superjumbo jet?
  2. What is the mean distance from the earth to the moon?
  3. In what year did the Russians launch Sputnik?
  4. In what year did Alaric lead the Visigoths in the Sack of Rome?
  5. How many career home runs did baseball giant Babe Ruth hit?
  6. How many iPhones did Apple sell in FY 2007, its year of introduction?
  7. How many transistors were on a 1993 Intel Pentium CPU chip?
  8. How many sheep were in New Zealand on 30 June 2006?
  9. What is the USGA-regulated minimum diameter of a golf ball?
  10. How tall is Victoria Falls on the Zambezi River?

Be sure to write down your answers (otherwise it’s too easy to rationalize ex post). No googling!

Answers at the end of next week.

*Update: edited slightly for greater clarity.

Thinking systemically about safetey

Accidents involve much more than the reliability of parts. Safety emerges from the systemic interactions of devices, people and organizations. Nancy Leveson’s Engineering a Safer World (free pdf currently at the MIT press link, lower left) picks up many of the threads in Perrow’s classic Normal Accidents, plus much more, and weaves them into a formal theory of systems safety. It comes to life with many interesting examples and prescriptions for best practice.

So far, I’ve only had time to read this the way I read the New Yorker (cartoons first), but a few pictures give a sense of the richness of systems perspectives that are brought to bear on the problems of safety:

Leveson - Pharma safety
Leveson - Safety as control
Leveson - Aviation information flow
The contrast between the figure above and the one that follows in the book, showing links that were actually in place, is striking. (I won’t spoil the surprise – you’ll have to go look for yourself.)

Leveson - Columbia disaster

Minds are like parachutes, or are they dumpsters?

Open Minds has yet another post in a long series demolishing bizarre views of climate skeptics, particularly those from WattsUpWithThat. Several of the targets are nice violations of conservation laws and bathtub dynamics. For example, how can you believe that the ocean is the source of rising atmospheric CO2, when atmospheric CO2 increases by less than human emissions and ocean CO2 is also rising?

The alarming thing about this is that, if I squint and forget that I know anything about dynamics, some of the rubbish sounds like science. For example,

The prevailing paradigm simply does not make sense from a stochastic systems point of view – it is essentially self-refuting. A very low bandwidth system, such as it demands, would not be able to have maintained CO2 levels in a tight band during the pre-industrial era and then suddenly started accumulating our inputs. It would have been driven by random events into a random walk with dispersion increasing as the square root of time. I have been aware of this disconnect for some time. When I found the glaringly evident temperature to CO2 derivative relationship, I knew I had found proof. It just does not make any sense otherwise. Temperature drives atmospheric CO2, and human inputs are negligible. Case closed.

I suspect that a lot of people would have trouble distinguishing this foolishness from sense. In fact, it’s tough to precisely articulate what’s wrong with this statement, because it falls so far short of a runnable model specification. I also suspect that I would have trouble distinguishing similar foolishness from sense in some other field, say biochemistry, if I were unfamiliar with the content and jargon.

This reinforces my conviction that words are inadequate for discussing complex, quantitative problems. Verbal descriptions of dynamic mental models hide all kinds of inconsistencies and are generally impossible to reliably test and refute. If you don’t have a formal model, you’ve brought a knife, or maybe a banana, to a gunfight.

There are two remedies for this. We need more formal mathematical model literacy, and more humility about mental models and verbal arguments.

Computational gains in complex modeling

Interesting approaches to crowd simulation by abstracting agents to fluid fields (around 6:20), and model reduction for fast simulation of high-dimensional fluid problems (around 23:00) and realtime control (33:00):

I haven’t really digested the implications of this, but it’s interesting to consider what the implications might be for simulating lumpier systems, like traditional SD or economic models, where model reduction has not been very widespread, or for large-scale computing like climate models.

Reading between the lines

… on another incoherent Breakthrough editorial:

The Creative Destruction of Climate Economics

In the 70 years that have passed since Joseph Schumpeter coined the term “creative destruction,” economists have struggled awkwardly with how to think about growth and innovation. Born of the low-growth agricultural economies of 18th Century Europe, the dismal science to this day remains focused on the question of how to most efficiently distribute scarce resources, not on how to create new ones — this despite two centuries of rapid economic growth driven by disruptive technologies, from the steam engine to electricity to the Internet.

Perhaps the authors should consult the two million references on Google scholar to endogenous growth and endogenous technology, or read some Marx. Continue reading “Reading between the lines”

Facebook reloaded

Facebook trading opened with it’s IPO and closed at $105 billion market capitalization.

I wondered how my model tracked reality over the last six months.

Facebook stats put users at 901 million at the end of March. My maximum likelihood run was rather lower than that – it corresponds with the K950 run in my last post (saturation users of 950 million), and predicted 840M users for end of Q1 2012. The latest data point corresponds with my K1250 run. I’m not sure if it’s interesting or not, but the new data point is a bit of an outlier. For one thing, it’s reported to the nearest million at a precise time, not with aggressive rounding as in earlier numbers I’d found. Re-estimating the model with the new, precise data point, it’s necessary to pass on the high size over most of the data from 2008-2011. That seems a bit fishy – perhaps a change in reporting methods has occurred.

In any case, it hardly matters whether the user carrying capacity is a bit over or under a billion. Either way, the valuation with current revenue per user is on the order of $20 billion. I had picked $5/user/year based on past performance, which turned out to be very close to the 2011 actuals. It would take a 10-year ramp to 7x current revenue/user to justify current pricing, or very low interest rates and risk premiums.

So the real question is, can Facebook increase its revenue per user dramatically?

Another short sell opportunity?

“I have no interest in shorting a cultural phenomenon,” hedge fund manager Jeffrey Matthews of Ram Partners in Greenwich, Connecticut, told Reuters in an email interview.

Asked if this was because such stocks trade without regard to normal market valuation, he wrote back, “Bingo.”

What drives learning?

Sit down and shut up while I tell you.

One interesting take on this compares countries cross-sectionally to get insight into performance drivers. A colleague dug up Educational Policy and Country Outcomes in International Cognitive Competence Studies. Two pictures from the path analysis are interesting:

Note the central role of discipline. Interestingly, the study also finds that self-report of pleasure reading is negatively correlated with performance. Perhaps that’s a consequence of getting performance through discipline rather than self-directed interest? (It works though.)

More interesting, though, is that practically everything is weak, except the educational level of society – a big positive feedback.

I find this sort of analysis quite interesting, but if I were a teacher, I think I’d be frustrated. In the aggregate international data, there’s precious little to go on when it comes to deciding, “what am I going to do in class today?”

 

 

Politicians designing control systems, badly

We already have to fly in planes designed by lawyers (metaphorically speaking). Now House Republicans want to remove the windows and instruments from the cockpit. This is stupid. Really stupid. I’ve used ACS data on numerous public and private sector consulting engagements. I’m perfectly willing to pay for the data, but I seriously doubt that the private sector will supply a substitute. Anyway, some basic free data is needed so that all citizens can participate intelligently in democracy. Lacking that, we’ll have to fly blind. Say, what’s a mountain goat doing up here in a cloud bank?