Bayesian epistemology cannot be done by humans. Do not trust any human who talks like it can.

[reddragdiva]

I started reading LessWrong in 2010 and they went on about "Bayesianism". The hot new epistemology that handles everything!

They explain it for single numbers well. This is a legitimately cool web page explaining Bayes' theorem with little JavaScript calculators, and is Eliezer Yudkowsky at his best as a science populariser.

Rationalists would go on about Bayesing your beliefs — update on information!

But probabilities are distributions, not single numbers. So the subculture is supplying an illusion of understanding here, not the actual understanding.

The rationalist subculture's entire understanding of statistics is "Bayes good, frequentist evil." And they can't do the numbers for Bayes either. Because the numbers for Bayes are hard.

This is what Elizabeth Sandifer calls "literary Bayesianism" in Neoreaction a Basilisk. Literary Bayesianism is so much easier than the kind with actual numbers in.

When I was still reading LessWrong, I went "OK, this Bayes thing sounds way cool, I'll take the first obvious step and read a book or something, this [I forget which book] is a recommended starter textbook ..." (starts book) "holy Jesus fuck what the"

(these people have, of course, never done the reading either, beyond alluding to Jaynes and certainly not reading all 758 pages)

Like, of course priors are distributions, not single numbers. And not nice well-behaved distributions you have an equation for — no, they're weird lumpy irregular shit with normal bits in maybe.

You're not talking about running a number through an equation — you're talking about running a couple of matrices through a transformation.

Once you realise priors are lumpy distributions, literary "Bayesianism" promptly falls apart, as a word game that 0 of its perpetrators are using anything resembling numbers for when they say "I updated on that." Like most Yudkowskian advances, it's just a verbal sketch of what their claimed solution might look like.

(Maybe I'm being unfair here, but even the SEP description of Bayesian epistemology reads like a verbal sketch of how it might work in a hypothetical world where humans could do that, rather than a thing you could actually do in your real life.)

I have had rationalists tell me that pulling a number out of their arse and running it through Bayes makes it somehow more reliable than just pulling it out of their arse. Handmade bespoke artisanal bias laundering. Garbage in, LessWrong out.

How literary Bayesianism works in practice:

• I have Bayesian priors
• YOU have cognitive biases
• THEY are toxoplasmotic SJW filth

Bayesian means the ingroup can do no wrong and the outgroup can do no right, and the more gooder the ingroup and badder the outgroup the more Bayesian it is.

Comments

[naath]

http://www.inference.org.uk/mackay/itila/book.html

I follow... the easy bit, which is probability on the scale of "there are 10 red balls and 50 green balls", but get lost long before I could tell you how to replace a T test.

I had a low opinion of stats before I took his class, a lower one after, and a REALLY low one after seeing how any of it is used in the 'real' world of science... but replacing a bunch of poorly understood frequentist algorithms with poorly understood Bayesian ones, is in no way the answer to the world's problems.

[siderea]

Yes.

I was planning on writing a thing about the problem with what you term literary Bayesianism, coming to the conclusion you express so felicitously as "Handmade bespoke artisanal bias laundering" (I may have to gank that), when I was rudely interrupted by a global health catastrophe. Except instead of going after the Rationalists, I was planning on going after their prophets, Kahneman and Tversky, which seems to be where this idiocy actually comes from.

[reddragdiva]

They didn't actually encourage this sort of thing themselves, did they? At least Kahneman admitted the replication crisis had hit some of his work.

[siderea]

Yeah, it's in Thinking Fast and Slow. And IIRC, I found it in the intro of (at least) one of their papers.

This isn't replication crisis stuff. This is... one of the other bad things scientists do, using their descriptive research prescriptively.

(As a clinician, it's literally my job to use the descriptive results of research prescriptively, and when non-clinicians do that, they usually manage to do so in ways that are ghastly wrong, and also, from my perspective, glariingly obviously wrong, and slightly, or a lot, infuriating. But that's another long rant.)

[vampwillow]

I remember you telling me about LessWrong years ago, and taking a look at it. I think I stopped being interested in it not very long afterwards. Is it still there? (should I care?)

[reddragdiva]

sort of, and absolutely not

[redbird]

The not-actually-calculated form of Bayesianism that seems useful to me is the reminder that statistics aren't calculated in a vacuum. For example, for the hypothesis "the moon landing was faked" to be true, other things that would also have to be true include that the USSR would pretend to have lost the space race, and that large numbers of people could keep a secret for years, for no apparent benefit to themselves.

That may be so much Bayesian priors as Bayesian consequences: if X remedy sold as homeopathic works, either it's working for non-homeopathic reasons (or a lot of the rest of medicine and science has to be reexamined as well. And most of the time people aren't going back to those first principles, they're saying "there's no evidence this works, and lots of reasons to think it doesn't, one of which is that the postulated mechanism makes no sense and would require us to reconsider a lot of other science." But they didn't start at "here's why Avogadro's number matters" rather than "why aren't the homeopathic remedies that people pour down the sink affecting the biosphere?"