Month: January 2011

Rendering Data

Rendering is a process that converts waste animal tissue into stable, value-added materials. Rendering can refer to any processing of animal byproducts into more useful materials, or more narrowly to the rendering of whole animal fatty tissue into purified fats like lard or tallow. …

The majority of tissue processed comes from slaughterhouses, but also includes restaurant grease and butcher shop trimmings. This material can include the fatty tissue, bones, and offal, as well as entire carcasses of animals condemned at slaughterhouses, and those that have died on farms (deadstock), in transit, etc. …

Converting data from public sources and business archives into quality material, suitable for use in modeling, is often equally arduous if not quite so disgusting. Just substitute “back office systems” for “slaughterhouses,” “basement paper archives” for “fatty tissue, bones, and offal,” etc. Electronic systems create vast quantities of the stuff, but much of it could be aptly described as “deadstock.” The problem is that people can easily tolerate some dreck in material that they’re just browsing for information, so systems don’t go to great lengths to eliminate it. Models are pickier – a few extra zeroes somewhere can really affect calibration, for example.

There’s an interesting new weapon in the war against arcane formats, inconsistent field coding and other data flaws, google refine. It’s apparently a Freebase spinoff. I’ve only used it for one task so far: grouping ad hoc text in a column, recognizing that “Ventana Systems,” “Ventana Systems, Inc.,” and “Ventanna Systems” all mean the same thing. It has useful filtering and clustering tools that largely automate such painful manual tasks. It can also do something else I’ve often hungered for in the past: transform an indented list into a table format. I suspect there’s much more depth that I haven’t even seen. Best of all, it’s fairly easy to get started.

It won’t cure all sins of corporate data management, like throwing away everything more than a few years old, but I’ll definitely reach for it next time I have a platefull of messy data to clean up for a model. Check it out.

No animals were harmed in the writing of this post.

Deeper Lessons

From the mailbag, regarding my last post on storytelling and playing with systems,

I read your blog post from the 19th and wondered how you would compare what was presented in the blog in contrast with what Forrester said on on pg 17, “Deeper Lessons” in the paper at

http://sysdyn.clexchange.org/sdep/papers/D-4434-3.pdf

That paper is Jay Forrester’s 1994 Learning Through System Dynamics as Preparation for the 21st Century. There’s a lot of good thinking in it. Unfortunately, the pdf is protected, so I have to give you a screenshot:

Forrester 4434 excerpt

The “important implications” that might be missed are things like, “we cause our own problems,” the notion that cause and effect are separated in time and space, and the differences between high- and low-leverage policies. (Go read the original for more.)

I see the blog and paper as complementary. Forrester’s deeper learnings are things that emerge from understanding the way things work, and that understanding – he argues – is developed through experimentation. This is also the rationale for management flight simulators and other games that teach systems principles. I think the guidance toward important implications that Forrester advocates is not much different than the kind of reporting the blog seeks – coverage that illuminates system structure and its consequences.

I don’t think stories per se are the problem. Sometimes they do degenerate into the equivalent of a bad history textbook – a litany of he-said-she-said opinions and events without any organizing structure. However,  a story can be crafted to reveal the way things work, and systems thinkers often advocate the use of stories to present system insights. Perhaps we should be more cautious about that.

I think it’s very natural to drop from an operational description of a system to stories that are so much about people and events that they lose track of structure. For example, the article on the steam engine at howstuffworks, which ought to be structural if anything is, starts off with, “They were first invented by Thomas Newcomen in 1705, and James Watt (who we remember each time we talk about “60-watt light bulbs” and the such) made big improvements to steam engines in 1769.” If it’s hard for steam engines, which are well-understood, imagine how hard it is for a reporter to get beyond the words of a controversial topic like health care, where even experts are likely to ambiguous and conflicting mental models.

The cautionary aspect of stories reminded me of a section in The Fifth Discipline, about what happens when you don’t convey systemic understanding:

Unfortunately, much more common are leaders who have a sense of purpose and genuine vision, but little ability to foster systemic understanding. Many greate “charismatic” leaders, despite having a deep sense of purpose and vision, manage almost exclusively at the level of events. Such leaders deal in visions and crises, and little in between. They foster a lofty sense of purpose and mission. They create tremendous energy and enthusiasm. But, under their leadership, an organization caroms from crisis to crisis. Eventually, the worldview of people in the organization becomes dominated by events and reactiveness. People experience being jerked continually from one crisis to another; they have no control over their time, let alone their destiny. Eventually, this will breed deep cynicism about the vision, and about visions in general. The soil within which a vision must take root – the belief that we can influence our future – becomes poisoned.

Such “visionary crisis managers” often become tragic figures. Their tragedy stems from the depth and genuineness of their vision. They often are truly committed to noble aspirations. But noble aspirations are not enough to overcome systemic forces contrary to the vision. As the ecologists say, “Nature bats last.” Systemic forces will win out over the most noble vision if we do not learn how to recognize, work with, and gently mold those forces.