Limits to Big Data

I’m skeptical of the idea that machine learning and big data will automatically lead to some kind of technological nirvana, a Star Trek future in which machines quickly learn all the physics needed for us to live happily ever after.

First, every other human technology has been a mixed bag, with improvements in welfare coming along with some collateral damage. It just seems naive to think that this one will be different.

These are not the primary problem.

Second, I think there are some good reasons to think that problems will get harder at the same rate that machines get smarter. The big successes I’ve seen are localized point prediction problems, not integrated systems with a lot of feedback. As soon as causality are separated in time and space by complex mechanisms, you’re into sloppy systems territory, where data may constrain only a few parameters at a time. Making progress in such systems will increasingly require integration of multiple theories and data from multiple sources.

People in domains that have made heavy use of big data increasingly recognize this: Continue reading “Limits to Big Data”

The Nordhaus Nobel

Congratulations to William Nordhaus for winning a Nobel in Economics for work on climate. However … I find that this award leaves me conflicted. I’m happy to see the field proclaim that it’s optimal to do something about climate change. But if this is the best economics has to offer, it’s also an indication of just how far divorced the field is from reality. (Or perhaps not; not all economists agree that we have reached a Neoclassical nirvana.)

Nordhaus was probably the first big name in economics to tackle the problem, and has continued to refine the work over more than two decades. At the same time, Nordhaus’ work has never recommended more than a modest effort to solve the climate problem. In the original DICE model, the optimal policy reduced emissions about 10%, with a tiny carbon tax of $10-15/tonC – a lot less than a buck a gallon on gasoline, for example. (Contrast this perspective with Stopping Climate Change Is Hopeless. Let’s Do It.)

Nordhaus’ mild prescription for action emerges naturally from the model’s assumptions. Ask yourself if you agree with the following statements:

If you find yourself agreeing, congratulations – you’d make a successful economist! All of these and more were features of the original DICE and RICE models, and the ones that most influence the low optimal price of carbon survive to this day. That low price waters down real policies, like the US government’s social cost of carbon.

In any case, you’re not off the hook; even with these rosy assumptions Nordhaus finds that we still ought to have a real climate policy. Perhaps that is the greatest irony here – that even the most Neoclassical view of climate that economics has to offer still recommends action. The perspective that climate change doesn’t exist or doesn’t matter requires assumptions even more contorted than those above, in a mythical paradise where fairies and unicorns cavort with the invisible hand.

Dynamics of Dictatorship

I’m preparing for a talk on the dynamics of dictatorship or authoritarianism, which touches on many other topics, like polarization, conflict, terror and insurgency, and filter bubbles. I thought I’d share a few references, in the hope of attracting more. I’m primarily interested in mathematical models, or at least conceptual models that have clearly-articulated structure->behavior relationships. Continue reading “Dynamics of Dictatorship”

Ad Experiment

In the near future I’ll be running an experiment with serving advertisements on this site, starting with Google AdSense.

This is motivated by a little bit of greed (to defray the costs of hosting) and a lot of curiosity.

  • What kind of ads will show up here?
  • Will it change my perception of this blog?
  • Will I feel any editorial pressure? (If so, the experiment ends.)

I’m generally wary of running society’s information system on a paid basis. (Recall the first deadly sin of complex system management.) On the other hand, there are certainly valid interests in sharing commercial information.

I plan to write about the outcome down the road, but first I’d like to get some firsthand experience.

What do you think?

Update: The experiment is over.

AI is killing us now

I’ve been watching the debate over AI with some amusement, as if it were some other planet at risk. The Musk-Zuckerberg kerfuffle is the latest installment. Ars Technica thinks they’re both wrong:

At this point, these debates are largely semantic.

I don’t see how anyone could live through the last few years and fail to notice that networking and automation have enabled an explosion of fake news, filter bubbles and other information pathologies. These are absolutely policy relevant, and smarter AI is poised to deliver more of what we need least. The problem is here now, not from some impending future singularity.

Ars gets one point sort of right:

Plus, computer scientists have demonstrated repeatedly that AI is no better than its datasets, and the datasets that humans produce are full of errors and biases. Whatever AI we produce will be as flawed and confused as humans are.

I don’t think the data is really the problem; it’s the assumptions the data’s treated with and the context in which that occurs that’s really problematic. In any case, automating flawed aspects of ourselves is not benign!

Here’s what I think is going on:

AI, and more generally computing and networks are doing some good things. More data and computing power accelerate the discovery of truth. But truth is still elusive and expensive. On the other hand, AI is making bullsh!t really cheap (pardon the technical jargon). There are many mechanisms by which this occurs:

These amplifiers of disinformation serve increasingly concentrated wealth and power elites that are isolated from their negative consequences, and benefit from fueling the process. We wind up wallowing in a sea of information pollution (the deadliest among the sins of managing complex systems).

As BS becomes more prevalent, various reinforcing mechanisms start kicking in. Accepted falsehoods erode critical thinking abilities, and promote the rejection of ideas like empiricism that were the foundation of the Enlightenment. The proliferation of BS requires more debunking, taking time away from discovery. A general erosion of trust makes it harder to solve problems, opening the door for opportunistic rent-seeking non-solutions.

I think it’s a matter of survival for us to do better at critical thinking, so we can shift the balance between truth and BS. That might be one area where AI could safely assist. We have other assets as well, like the explosion of online learning opportunities. But I think we also need some cultural solutions, like better management of trust and anonymity, brakes on concentration, sanctions for lying, rewards for prediction, and more time for reflection.

Privatizing Public Lands – Claim your 0.3 acres now!

BLM Public Lands Statistics show that the federal government holds about 643 million acres – about 2 acres for each person.

But what would you really get if these lands were transferred to the states and privatized by sale? Asset sales would distribute land roughly according to the existing distribution of wealth. Here’s how that would look:

The Forbes 400 has a net worth of $2.4 trillion, not quite 3% of US household net worth. If you’re one of those lucky few, your cut would be about 44,000 acres, or 69 square miles.

Bill Gates, Jeff Bezos, Warren Buffet, Mark Zuckerberg and Larry Ellison alone could split Yellowstone National Park (over 2 million acres).

The top 1% wealthiest Americans (35% of net worth) would average 70 acres each, and the next 19% (51% of net worth) would get a little over 5 acres.

The other 80% of America would split the remaining 14% of the land. That’s about a third of an acre each, which would be a good-sized suburban lot, if it weren’t in the middle of Nevada or Alaska.

You can’t even see the average person’s share on a graph, unless you use a logarithmic scale:


Otherwise, the result just looks ridiculous, even if you ignore the outliers:


Tax cuts visualized

Much has been made of the fact that Trump’s revised tax plan cuts its implications for deficits in half (from ten to five trillion). Oddly, there’s less attention to the equity implications, which border on the obscene. Trump’s plan gives the top bracket a tax cut ten times bigger (as percentage of income) than that given to the bottom three fifths of the income distribution.

That makes the difference in absolute $ tax cuts between the richest and poorest pretty spectacular – a factor of 5000 to 10,000:


Trump tax cut distribution, by income quantile.

To see one pixel of the bottom quintile’s tax cut on this chart, it would have to be over 5000 pixels tall!

For comparison, here are the Trump & Clinton proposals. The Clinton plan proposes negligible increases on lower earners (e.g., $4 on the bottom fifth) and a moderate increase (5%) on top earners:


Trump & Clinton tax cut distributions, by income quantile.


Footing the bill for Iraq

Back in 2002, when invasion of Iraq was on the table and many Democrats were rushing patriotically to the President’s side rather than thinking for themselves, William Nordhaus (staunchest critic of Limits) went out on a limb a bit to attempt a realistic estimate of the potential cost.

All the dangers that lead to ignoring or underestimating the costs of war can be reduced by a thoughtful public discussion. Yet neither the Bush administration nor the Congress – neither the proponents nor the critics of war – has presented a serious estimate of the costs of a war in Iraq. Neither citizens nor policymakers are able to make informed judgments about the realistic costs and benefits of a potential conflict when no estimate is given.

His worst case: about $755 billion direct (military, peacekeeping and reconstruction) plus indirect effects totaling almost $2 trillion for a decade of conflict and its aftermath.

NordhausIraqNordhaus’ worst case is pretty close to actual direct spending in Iraq to date. But with another trillion for Afghanistan and 2 to 4 in the pipeline from future obligations related to the war, the grand total is looking like a lowball estimate. Other pre-invasion estimates, in the low billions, look downright ludicrous.

Recent news makes Nordhaus’ parting thought even more prescient:

Particularly worrisome are the casual promises of postwar democratization, reconstruction, and nation-building in Iraq. The cost of war may turn out to be low, but the cost of a successful peace looks very steep. If American taxpayers decline to pay the bills for ensuring the long-term health of Iraq, America would leave behind mountains of rubble and mobs of angry people. As the world learned from the Carthaginian peace that settled World War I, the cost of a botched peace may be even higher than the price of a bloody war