I promised some posts about the significance of Polish spaces in probability. I thought I would start with a philosophical point about the interpretation of probability.
Probability has a strange dual nature. Ask a mathematician, and you’ll get an answer in terms of measure theory. Ask someone who applies probability like a physicist or a statistician, and you’ll get an answer in terms of random draws generated by some physical process. But the two notions are the same, right? Not quite.
The measure-theoretic axioms of probability do not fully capture the folk intuition for continuous random variables. Measure-theoretic probability is not just more general, but it is missing one ingredient in what we mean by probability. That missing ingredient is supplied by the setting of complete metric spaces.
For continuous random variables, not all measure zero events are created equal. Suppose you have a random variable that is uniformly distributed between 0 and 1. Mathematically, the probability that the variable takes on the value of 0.5 and the value of 2 are the same: zero. But conceptually, when you draw from this random variable, some value between 0 and 1 actually happens (even though any specific value is very unlikely), and 2 never happens.
So what’s the difference? You can’t reach 0.5 itself with positive probability, but you do reach every neighborhood of 0.5 with positive probability. On the other hand, small enough neighborhoods of 2 occur with probability zero.
This points us to a method for interpreting the notion of drawing a point from a complete metric space. Imagine that after a random draw, we can ask for each open ball in the space whether an event occurred in that ball. To find out if a specific point occurred we check each open ball around that point to see if that ball occurred. To find out which point has occurred, we just need to find a sequence of open balls that contain the event whose radius go to zero. (By completeness, the intersection of these open balls describe a unique point.)
If the metric space is separable, we can extend this to give a method for simulating draws on a computer. For a fixed radius, we can cover the space by a countable number of open balls of that radius. (This claim isn’t completely obvious, but a standard result of point-set topology is that for a separable metric space every open cover of the space has a countable subcover. This is known as the Lindelöf property.) We randomly draw one of these open balls. Then, using that open ball as our new space, we repeat the process with a new ball of half the radius. After enough steps of this process, we have specified – up to an arbitrarily small error – a point from the space.
So separable metric spaces are a natural setting for probability, one that bridges the gap between the abstract notion of a probability space, and the concrete notion of physically taking a random draw. In a future post I will talk about some of the mathematical implications of this setting.
Actually if you ask some physicist or some statistician, you may hear that probability is about degrees of belief about propositions.
To get the difference between a “possible” probability zero event (0.5 in your example) and an “impossible” probability zero event (2 in your example) it doesn’t seem as though we need as much as a complete metric space. If we only had a topological space, we can get the same distinction by saying an event is possible if every open set containing that event has positive probability (i.e. possible events are those in the support of the probability measure).
It would be interesting for someone to write a book that pans back and forth rapidly between the sigma algebra view of probability and the intuitive view. Most books focus on one view and give footnotes about the other.
Along those lines, I’d also like to see a differential geometry book that frequently shifts perspectives — sophomore calculus, differential forms, tensor index juggling, etc — giving each roughly equal time, a sort of Rosetta stone.
To make my objection as short as possible: Comparing the “neighborhoods” (an inherently ambiguous notion) of 0.5 and 2 is no better than comparing 0.5 and 2 themselves. You’ve just shifted the question elsewhere!
Also, I’m inclined to agree with Valter that many, especially statisticians, would define probability with respect to epistemology and prediction, not in a descriptive way about entities.
As a man with only a mathematics minor I greatly appreciate these philosophical descriptions that set the stage. It really helps put things into perspective.
I thought Kolmogorov already had the technology to draw this distinction without mentioning neighborhoods or the like - we just have to specify the space that the probability distribution is over. There is a space that just contains the points in [0,1] that is uniform over them, and there is another space that contains all real numbers, but still has its probability distributed uniformly on [0,1], with any measurable set disjoint from [0,1] getting probability 0. On the former, 2 is impossible, while on the latter it is possible, but has probability 0, and in fact every neighborhood has probability 0.
Also, Kolmogorov’s theory draws the distinction without requiring that we add any additional topological structure to the space.
There are more than two interpretations of probability. Perhaps this is because our intuitions on the subject are partly driven by our intended applications, and application domains (despite the claims of ideologues) differ markedly from one another.
For anyone interested in some of the other interpretations, I suggest reading Donald Gillies’ nice book, “Philosophical Theories of Probability” (London, UK: Routledge, 2000).
Kenny: I claim that the Kolmogorov axioms do not fully capture the folk intuition of probability, and this is something you can see if a couple of different ways. If you don’t buy this example (and to my personal intuition of what probability means, 2 is literally impossible, while any specific number between 0 and 1 is possible, but probability zero; this is obviously subjective) I have some more posts planned to explain what I mean.
I think I agree with you that Kolmogorov doesn’t deal with all the intuitive issues - I’m just suggesting that this isn’t one of those issues. I’m definitely interested in seeing the future posts!
Regarding an Axiomatic ( measure theory ) vs. an Intuitionistic approach to probability, I am reminded of reading the text by DeFinetti.
As I recall, DeFinetti pursued an approach that did not use measure theory. He seemed to be making a notion of expectation as the foundation for constructing a theory of probability.
As I read his text, I got the impression that he was hand-waving his way around what were essentially measure theoretic concepts. In avoiding measure theory, he was adding confusion to the subject.
IMHO, probability is best viewed as a discipline in applied measure theory or perhaps more precisely, computational measure theory. In this view, separable metric spaces would appear to be necessary. I don’t know if one can compute anything without at least having a separable metric space at hand.