In {R^n, n large}, no one can hear you scream.

I haven’t had time to write much lately. I spent several weeks in arcane code purgatory, discovering the fun of macros containing uninitialized thread pointers that only fail in 64 bit environments, and for different reasons on Windows, Mac and Linux. That’s a dark place that I hope never again to visit.

Now I’m working fun things again, but they’re secret, so I can’t discuss details. Instead, I’ll just share a little observation that came up in the process.

Frequently, we do calibration or policy optimization on models with a lot of parameters. “A lot” is actually a pretty small number – like 10 – when you have to do things by brute force. This works more often than we have a right to expect, given the potential combinatorial explosion this entails.

However, I suspect that we (at least I) don’t fully appreciate what’s going on. Here are two provable facts that make sense upon reflection, but weren’t part of my intuition about such problems:

In other words, R^n gets big really fast, and it’s all corners. The saving grace is probably that sensible parameters are frequently distributed on low-dimensional manifolds embedded in high dimensional spaces. But we should probably be more afraid than we typically are.

2 thoughts on “In {R^n, n large}, no one can hear you scream.”

  1. That sounds a bit like Michael Betancourt’s description of typical sets in the realm of Hamiltonian Monte Carlo. Thought provoking, isn’t it?

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.