I ran across this entry, the other day, in which Gene Callahan takes issue with Bayesian inference, as described in the last post:
the basic idea is that you "start out" by assigning some "prior probabilities" to various theories about some phenomenon, or outcomes of some event, and then multiply those "priors" by a factor based on how much more or less likely new evidence makes the prior.
For instance, you are a late 19th century physicist, and you are evaluating how likely it is the Newtonian mechanics is the true description of matter in motion. At that time, there would have been physicists who would assign p=1 to it being true, and p=0 to it being false. At the very least, many physicists would have assigned p=0 to something as weird as quantum mechanics being true!
Now, as the years pass, you are presented with startling new evidence about black body radiation, the photoelectric effect, and so on, and with a startling new theory in addition. According to the theory of Bayesian updating, the "rational" response is just to think you must be delusional in believing you have heard this new data! You had assigned an alternative theory a prior of 0, and now no factor the new evidence recommends multiplying that prior by can ever change that initial assignment of p=0.
Of course, that is not what real scientists did at all. Instead, they assigned whole new "priors" -- they thought, "Mon Dieu, I had never considered the possibility of this theory or this evidence, and therefore I was in a state of 'radical uncertainty,' and ought to re-think everything." But allowing that maneuver thwarts the whole motive for Bayesian updating, which is to turn rational choice between theories into a formal, mechanical procedure.
I have my own response to this, which I'll get to in a bit, but I noticed Callahan also leaves this message in the comments for someone who disagrees with him:
Oh, and anonymous, it's really best you leave this sort of thing to the experts, OK?
It was a response to something the anonymous poster said to the same effect, but in a far more arrogant context. So, yes, let's do exactly that. Let's find out what E.T. Jaynes, the expert of Bayesian probability, had to say, discussing the case of scientific experiments appearing to validate ESP:
[An ESP researcher] will then react with anger and dismay when, in spite of what he considers this overhwelming evidence, we persist in not believing in ESP. Why are we, as he sees it, so perversely illogical and unscientific?
The trouble is that the above calculations represent a very naïve application of probability theory, in that they consider only Hp and Hf; and no other hypotheses. If we really knew that Hp and Hf were the only possible ways the data (or more precisely, the observable report of the experiment and data) could be generated, then the conclusions that follow from [the above equations] would be perfectly all right. But in the real world, our intuition is taking into account some additional possibilities that they ignore.
Probability theory gives us the results of consistent plausible reasoning from the information that was actually used in our calculation. It can lead us wildly astray. . . if we fail to use all the information that our common sense tells us is relevant to the question we are asking. When we are dealing with some extremely implausible hypothesis, recognition of a seemingly trivial alternative possibility can make orders of magnitude difference in the conclusions. . .
There are, perhaps, 100 different deception hypotheses that we could think of and are not too far-fetched to consider, although a single one would suffice to make our point. . .
Introduction of the deception hypotheses has changed the calculation greatly. . . each of the hypotheses is, in my judgment, more likely than Hf, so there is not the remotest possibility that the inequality could ever be satisfied.
Therefore, this kind of experiment can never convince me of the reality of Mrs. Stewart's ESP: not because I assert Pf = 0 dogmatically at the start, but because the verifiable facts can be accounted for by many alternative hypotheses, every one of which I consider inherently more plausible than Hf, and none of which is ruled out by the information available to me
You can read the chapter for the probability calculations. The point is, there are many inequalities that can arise in Bayes' Theorem applications that look like unreasonable dogma. Sometimes they are and sometimes they are not. Physicists from the 19th century didn't dogmatically believe their theories with 100% probability. They simply took the evidence they had currently available to them and applied it. The fact that they were open to evidence at all means that their priors were not really 1 or 0! If you don't believe it, it's pretty easy to think of groups who do have priors of 1 or 0 for their beliefs. Young Earth Creationists can be presented with all the evidence in the world for an old earth, but most of them will never be convinced. 9/11 truthers can examine the evidence for their grand questions all day, and never arrive at the right answer.
The secret to science is this: you never assign a probability of 0 or 1 to a proposition. Even if you think you're doing so, as long as you're willing to believe something else, your brain is revising that probability you think you have slightly downward.. What follows is that, likewise, human argument is about concentrating probability mass. This is where so many logical fallacies come from, it's why politicians are miserable to listen to, and it's why Smith, from my modified example in the last entry, is a very foolish person for assigning an exactly equal probability to all alternate, non-cigarette hypotheses. If the reverse can't be true, it's not an argument. If a statement doesn't give us something to plug into a Bayesian framework, no matter if it can only vaguely approximate the mathematical calculations, it's effectively meaningless.