Complete Metric Spaces and the Interpretation of Probability

I promised some posts about the significance of Polish spaces in probability. I thought I would start with a philosophical point about the interpretation of probability.

Probability has a strange dual nature. Ask a mathematician, and you’ll get an answer in terms of measure theory. Ask someone who applies probability like a physicist or a statistician, and you’ll get an answer in terms of random draws generated by some physical process. But the two notions are the same, right? Not quite.

The measure-theoretic axioms of probability do not fully capture the folk intuition for continuous random variables. Measure-theoretic probability is not just more general, but it is missing one ingredient in what we mean by probability. That missing ingredient is supplied by the setting of complete metric spaces.

For continuous random variables, not all measure zero events are created equal. Suppose you have a random variable that is uniformly distributed between 0 and 1. Mathematically, the probability that the variable takes on the value of 0.5 and the value of 2 are the same: zero. But conceptually, when you draw from this random variable, some value between 0 and 1 actually happens (even though any specific value is very unlikely), and 2 never happens.

So what’s the difference? You can’t reach 0.5 itself with positive probability, but you do reach every neighborhood of 0.5 with positive probability. On the other hand, small enough neighborhoods of 2 occur with probability zero.

This points us to a method for interpreting the notion of drawing a point from a complete metric space. Imagine that after a random draw, we can ask for each open ball in the space whether an event occurred in that ball. To find out if a specific point occurred we check each open ball around that point to see if that ball occurred. To find out which point has occurred, we just need to find a sequence of open balls that contain the event whose radius go to zero. (By completeness, the intersection of these open balls describe a unique point.)

If the metric space is separable, we can extend this to give a method for simulating draws on a computer. For a fixed radius, we can cover the space by a countable number of open balls of that radius. (This claim isn’t completely obvious, but a standard result of point-set topology is that for a separable metric space every open cover of the space has a countable subcover. This is known as the Lindelöf property.) We randomly draw one of these open balls. Then, using that open ball as our new space, we repeat the process with a new ball of half the radius. After enough steps of this process, we have specified – up to an arbitrarily small error – a point from the space.

So separable metric spaces are a natural setting for probability, one that bridges the gap between the abstract notion of a probability space, and the concrete notion of physically taking a random draw. In a future post I will talk about some of the mathematical implications of this setting.