Tuesday, July 15, 2008

The secret to science

I ran across this entry, the other day, in which Gene Callahan takes issue with Bayesian inference, as described in the last post:

the basic idea is that you "start out" by assigning some "prior probabilities" to various theories about some phenomenon, or outcomes of some event, and then multiply those "priors" by a factor based on how much more or less likely new evidence makes the prior.

For instance, you are a late 19th century physicist, and you are evaluating how likely it is the Newtonian mechanics is the true description of matter in motion. At that time, there would have been physicists who would assign p=1 to it being true, and p=0 to it being false. At the very least, many physicists would have assigned p=0 to something as weird as quantum mechanics being true!

Now, as the years pass, you are presented with startling new evidence about black body radiation, the photoelectric effect, and so on, and with a startling new theory in addition. According to the theory of Bayesian updating, the "rational" response is just to think you must be delusional in believing you have heard this new data! You had assigned an alternative theory a prior of 0, and now no factor the new evidence recommends multiplying that prior by can ever change that initial assignment of p=0.

Of course, that is not what real scientists did at all. Instead, they assigned whole new "priors" -- they thought, "Mon Dieu, I had never considered the possibility of this theory or this evidence, and therefore I was in a state of 'radical uncertainty,' and ought to re-think everything." But allowing that maneuver thwarts the whole motive for Bayesian updating, which is to turn rational choice between theories into a formal, mechanical procedure.

I have my own response to this, which I'll get to in a bit, but I noticed Callahan also leaves this message in the comments for someone who disagrees with him:
Oh, and anonymous, it's really best you leave this sort of thing to the experts, OK?

It was a response to something the anonymous poster said to the same effect, but in a far more arrogant context. So, yes, let's do exactly that. Let's find out what E.T. Jaynes, the expert of Bayesian probability, had to say, discussing the case of scientific experiments appearing to validate ESP:
[An ESP researcher] will then react with anger and dismay when, in spite of what he considers this overhwelming evidence, we persist in not believing in ESP. Why are we, as he sees it, so perversely illogical and unscientific?

The trouble is that the above calculations represent a very naïve application of probability theory, in that they consider only Hp and Hf; and no other hypotheses. If we really knew that Hp and Hf were the only possible ways the data (or more precisely, the observable report of the experiment and data) could be generated, then the conclusions that follow from [the above equations] would be perfectly all right. But in the real world, our intuition is taking into account some additional possibilities that they ignore.

Probability theory gives us the results of consistent plausible reasoning from the information that was actually used in our calculation. It can lead us wildly astray. . . if we fail to use all the information that our common sense tells us is relevant to the question we are asking. When we are dealing with some extremely implausible hypothesis, recognition of a seemingly trivial alternative possibility can make orders of magnitude difference in the conclusions. . .

There are, perhaps, 100 different deception hypotheses that we could think of and are not too far-fetched to consider, although a single one would suffice to make our point. . .

Introduction of the deception hypotheses has changed the calculation greatly. . . each of the hypotheses is, in my judgment, more likely than Hf, so there is not the remotest possibility that the inequality could ever be satisfied.

Therefore, this kind of experiment can never convince me of the reality of Mrs. Stewart's ESP: not because I assert Pf = 0 dogmatically at the start, but because the verifiable facts can be accounted for by many alternative hypotheses, every one of which I consider inherently more plausible than Hf, and none of which is ruled out by the information available to me

You can read the chapter for the probability calculations. The point is, there are many inequalities that can arise in Bayes' Theorem applications that look like unreasonable dogma. Sometimes they are and sometimes they are not. Physicists from the 19th century didn't dogmatically believe their theories with 100% probability. They simply took the evidence they had currently available to them and applied it. The fact that they were open to evidence at all means that their priors were not really 1 or 0! If you don't believe it, it's pretty easy to think of groups who do have priors of 1 or 0 for their beliefs. Young Earth Creationists can be presented with all the evidence in the world for an old earth, but most of them will never be convinced. 9/11 truthers can examine the evidence for their grand questions all day, and never arrive at the right answer.

The secret to science is this: you never assign a probability of 0 or 1 to a proposition. Even if you think you're doing so, as long as you're willing to believe something else, your brain is revising that probability you think you have slightly downward.. What follows is that, likewise, human argument is about concentrating probability mass. This is where so many logical fallacies come from, it's why politicians are miserable to listen to, and it's why Smith, from my modified example in the last entry, is a very foolish person for assigning an exactly equal probability to all alternate, non-cigarette hypotheses. If the reverse can't be true, it's not an argument. If a statement doesn't give us something to plug into a Bayesian framework, no matter if it can only vaguely approximate the mathematical calculations, it's effectively meaningless.

11 comments:

silkop said...

This Gene Callahan guy seems to be an idiot. Not only does he put his utopian 0/1 probability into minds of some straw "scientists" just to accuse them (as you correctly noticed). He also blatantly ignores the fact that Bayesian probabilities are all conditional. When information on the right side of the vertical bar changes (new evidence - or new hypotheses to compare against - that's the "background information"!), so do the probabilities. Gene C. remains under the false impression that probabilities are properties of things (even abstract stuff like theories), where in fact they are a description of someone's state of information about things. Worse yet, he seems to assume that probability theory should/could give you ontologically true statements even if you put garbage into it.

I'm not posting a comment in his blog because he seems to reject corrections a priori (which just proves that he has no idea about Bayesian inference, BTW).

Gene Callahan said...

"If a statement doesn't give us something to plug into a Bayesian framework, no matter if it can only vaguely approximate the mathematical calculations, it's effectively meaningless."

What does the above statement give you to "plug into a Bayesian framework"?

Gene Callahan said...

"This Gene Callahan guy seems to be an idiot. Not only does he put his utopian 0/1 probability into minds of some straw "scientists" just to accuse them (as you correctly noticed)."

Read some history of science rather (I did graduate work in the field at King's College, btw) rather than just making up off the top of your head that no scientists ever assigned 1 or 0. At the end of the 19th century there were physicists saying physics was done and there was nothing new to discover in it!

"When information on the right side of the vertical bar changes (new evidence - or new hypotheses to compare against - that's the "background information"!), so do the probabilities."

Doh! Did you notice I was talking about Bayesian UPDATING! That this UPDATING of probabilities was exactly what I was speaking of? It's great to be called an idiot by someone who can whose reading comprehension is at about a 3rd grade level!

Swimmy Lionni said...

What does the above statement give you to "plug into a Bayesian framework"?

Perhaps best answered with another question: could any evidence convince you that Bayesian probability is more or less correct than traditional probability? The answer, in your case, is obviously yes; you view the history of science as evidence that Bayes is wrong. So Bayes' theorem itself can be plugged into a Bayesian framework.

If Bayes itself can be plugged into a Bayesian framework, then so can my original contention: We can use Bayes + evidence to update our probability estimations of the truth of, "Statements that can't be plugged into a Bayesian framework are meaningless."

As for the history of science issue: a prior isn't necessarily what you claim it to be. Look around the internet and you'll find lots of 9/11 Truthers, astrologers, etc. exclaim that they're as skeptic people as you can find, but they're not. You'll find creationists exclaiming that they've studied the evidence deeply and have kept an open mind, but they haven't. A scientist who is sure he's right about something but changes his mind based on new evidence doesn't have a prior of 1 or 0, no matter what he says.

Gene Callahan said...

"The answer, in your case, is obviously yes; you view the history of science as evidence that Bayes is wrong."

Good God man, can you or any of your commenters read at all? Where did I ever say Bayes was wrong? I contended that the idea of Bayesian updating was being misused. And I recently found evidence from Bayes himself as to how he intended his work to be used:
http://www.gene-callahan.org/blog/2008/08/updating-bayes.html

And why the heck didn't you call out "silkop" when he implied I didn't know that Batesian UPDATING implied, well, UPDATING -- just because he was praising you?

Swimmy Lionni said...

Gene, because silkop and I are both also talking about updating, though we mean updating many different hypotheses simultaneously based on the same evidence. That is, "Something else is going on entirely" is another hypothesis that the available evidence would update.

In the ball example, you use a prior of "The Central Limit Theorem" is true to arrive at the conclusion "the table is biased." Although, if all tables are similarly "biased" despite being measured as perfectly level, you may eventually update your CLT prior instead.
It's not possible to start with a blank slate, zero priors. For more on that line see here.

Gene Callahan said...

"Although, if all tables are similarly "biased" despite being measured as perfectly level, you may eventually update your CLT prior instead."

OK, yes, that was exactly what I was saying, contra Caplan -- sometimes it is the priors that need to be updated! So you agree with me here.

But "silkop" contended that I did not even know that "post-prior"-probabilities could be adjust, whereas I was saying "not only can they be adjusted, but so can the priors as well." So when "silkop" claims I think all probalities are objectively fixed, he's so far wrong that my point was that proabilities are even less fixed than Caplan would have them -- your "priors" may need to be updated as well, as you and seem to agree!

Gene Callahan said...

That was "as you and I seem to agree".

Gene Callahan said...

'That is, "Something else is going on entirely" is another hypothesis that the available evidence would update.'

Yes, yes, I realize that -- but, the point is this -- yes, per Caplan, you can assign any prior you want to "things I have never even imagined in my wildest dreams" -- and the formal Batesian framework will remain intact. But, per Kirzner (and me), such as assignment has no real meaning for human action -- yes, perhaps I could say. "There is a .1 probability that something "I have never even imagined in my wildest dreams" will take place -- but given that I have never even imagined such a thing, such a probability assignment means essentially diddly-squat.

Swimmy Lionni said...

Sorry for the long response time, Gene. I think there are at least two related implications:

1) Your priors in other areas will influence what new hypotheses you pick up. If you have a strong reductionist prior, quantum physics isn't going to lead you to suddenly believe in magic.

2) When you do pick a new hypothesis, contra traditional rationality, you should not pick just any hypothesis that is potentially testable. You should choose the one you assign the highest probability to.

Gene Callahan said...

"1) Your priors in other areas will influence what new hypotheses you pick up. If you have a strong reductionist prior, quantum physics isn't going to lead you to suddenly believe in magic."

OK.

"2) When you do pick a new hypothesis, contra traditional rationality, you should not pick just any hypothesis that is potentially testable. You should choose the one you assign the highest probability to."

Huh? The only person I know who ever said something like "you should pick just any hypothesis that is potentially testable" was Popper, who certainly does not represent "traditional rationality"!