Bayesian Probability? Convince me...

Not true. The idea of a determined universe is not disputable; it simply reflects the way we think. If a change in the way believed physical laws work was observed, it would not affect determinism. The changes would become a part of the new model of determination. In other words, determinism means nothing.

This kinda relates to the flaw that I found in Caplan’s argument against radical uncertainty. There is what I call a “cheat” often used in probability. If you buy the car and you find that it is a lemon whatever probabilities you previously used are now moot. There was/is no probability that the car is a lemon. It was/is a lemon. Whatever you imagined about what good the car could have been was no more knowledge than had you imagined it might sprout wings and fly away. The 30% and 70% enumerations are also meaningless. Whatever possibility you act on is 1 and the rest are 0.

Determinism and Indeterminism are meaningful epistemological positions. One is correct and the other is false.

Both are metaphysical hypotheses as Mises would say, about phenomena whose causal relations have not been established. Heisenberg’s position, who Crovelli claimed to be indirectly attacking by critiqueing R von Mises, really says nothing more than that we cannot honestly be anything more than agnostic about breaking down to more primitive causes phenomena we have established as ultimate givens until we do so.

An instructive example in the case of probabillity is QM. In contrast to statistical mechanics, the theory is inherently probabilistic, we cannot know anything more than the state vector of a QM particle or system. Taking the modulus squared of the state vector in a particular basis, e.g. the wavefunction produces an “amplitude” that is interpreted as a probabillity distribution of the entire set of outcomes if the same experiment is repeated over and over for single particle QM. This is the best we can do, in principle we cannot say more.

From my favourite quote in Theory and History (pg. 86-89):

Interesting, thanks, I hadn’t got round to reading Theory and History yet. I think the law of large numbers of course does allow you to extract away microscopic irregularities though this depends entirely on the nature of the distribution one is dealing with. For gaussian distributions; the most common in physics this is certainly true. If one wants to interpret the regularity produced on a macroscopic scale as therefore following from the microscopic scale for certain distributions, then at first this seems largely a matter of semantics, though it certainly seems a more reasonable assertion than the opposite.

Aside from that, I find his account highly agreeable, especially when he talks about the imperfect knowledge we have of the causal influence of factor A; this is precisely the message of Quantum Mechanics, we only know the outcomes of the set and that is the best we can do, even in principle(this is different to statistical mechanics which was deterministic structurally on a microscopic scale, but had to be interpreted statistically in a different way on a larger scale in order to extract the physical laws it implies. See Loschmidt’s Paradox and the Boltzmannn H theorem for more on that). He says the same thing in Human Action, which is just one of the things that resonated with me when reading it.

Finally, I must confess I am puzzled by Mises’ criticism of “eminent physicists,” since he has not specified by who they are; though if he means Niels Bohr and the proponents of the Copenhagen interpretation I think he has misunderstood them. These writers don’t challenge determinism, just say that the probabillity of a class of outcomes is in principle the best we can know of certain events, just as he does. This was opposed to writers like Einstein, who on top of other things disliked the lack of perfect determinism in Quantum Mechanics, an assumption that had also been assumed unquestionably by classical empircists and physicists.

I do know some physicists and even more so popularisers of physics do convey QM as “indeterministic”, so I can’t accuse Mises of mis-representation since I don’t know who he was criticising.

Determinism is not an episteme. It’s only assumption is that laws are constant, which really says nothing because inconstancy can be a law (indeterminism). It is correct by definition, not by reference, and nothing can be derived from it.

Well, the explanation is really, really involved, but I’ll give you the highlights.

Within computer science, there is a discipline called “computability theory” which concerns itself with identifying unsolvable problems, that is, problems for which no faster solution than “turn it on and see what happens” exists. Computability theory seeks to find the limits of what is computable at all.

Based on the results of computability theory - which are closely linked with Godel’s incompleteness theorems - the field of Algorithmic Information Theory, AIT, (or “Kolmogorov Information Theory” or “Kolomogorov-Chaitin Information Theory” or “Kolmogorov-Chaitin-Solomonoff Information Theory” … you see how absurd the attribution game can become) has emerged. The basic idea of AIT is the following: given a string* of symbols, S, what is the shortest program, p, which, when run on a computer, T, will output S? In symbols:

min |p| : T(p) = S (footnote**)

It turns out that there are all sorts of fascinating implications for mathematics from this simple idea, not least of which is Chaitin’s constant from which Chaitin draws the startling conclusion that “Some mathematical facts are true for no reason, they’re true by accident.” In fact, Chaitin doesn’t stop here, he even goes so far as to say that almost all mathematical facts are true for no reason. To construct his constant, Chaitin created a mathematical measure over the set of all input programs to a computer, using a prefix code. This measure is a formal expression of the intuitive principle of Ockham’s Razor - the simpler explanation (shorter program) should be preferred to the longer.

Solomonoff used a similar measure on the set of all input programs but he added another innovation. He asked the following question: Given a n-bit prefix of a string X, what do we expect the next bit of X to be if we have received n bits of X so far?*** To answer this question, Solomonoff proposed the following:

  1. Run every program of length < n and see what its output is
  2. For each program which outputs the first n-bits of X, see what the next bit is
  3. Sum the weights of the programs which output a ‘1’ according to the measure given above (do the same for ‘0’). This yields a probability for the next bit of X being a 1 or 0.

Let’s say that the 10-bit prefix of X is the binary sequence 0101010101. What do you, intuitively, expect the next bit of X to be? 0? or 1? I would guess 0 as would most people. But why?? Solomonoff tells us why - because the shortest (most heavily weighted) programs which output 0101010101 output a 0 as the next bit. Other programs might output a ‘1’, but they are longer and more complex, hence, they receive less “weight” (in keeping with Ockham’s principle of brevity).

It should be obvious, at this point, what the philosophical implications for the problem of induction (PoI) are. Seeing a regular pattern in the past does tell us something about the future if we believe that simpler hypotheses are more probable than more complex hypotheses (Ockham’s Razor). Solomonoff allows us to reduce the PoI to belief in Ockham’s Razor. If you believe OR, then the PoI is automatically solved. We can expect the future to be like the past because the simpler hypothesis always generalizes from the past to the future. Laws which only operate over certain spans of time are more complex, less probable, than laws which operate at all times.

Clayton -

  • A “string” is an ordered list of symbols drawn from a finite alphabet

** the bars mean “length of”, i.e. |X| reads “length of the string X” and is equal to the number of symbols in the string (repetitions count). If X = “abca”, then |X| = 4.

***There are other ways to ask this question and this is not exactly how Solomonoff approaches it, this is just the way I like to think about it

Those seem like methods for computing a posteriori probability to me. How does that give us a basis for a priori probability? And, of what use is a theory for calculating probability, if it rests on assumptions which unprovable?

Whatever it may seem like, Solomonoff’s Algorithmic Probability is, in fact, a measure of the a priori probability, or the “prior probability” in Bayesian terminology. It is an objective measure of the “probability of the hypothesis”, something which is typically treated as subjective or, at best, unknowable.

A posteriori probability does not but algorithmic probability is an a priori measure. Why don’t you explain what makes you think it is a posteriori and see if I can find the flaw in your reasoning? Also, read this awesome little introduction to the whole subject.

Um, every theory rests on assumptions which are unprovable.

Clayton -

Great explanation.

The problem with the AIT theory, however, is that the Razor is not a true way of making logical deductions. It is merely looking for the shortest program, no more, no less.

For instance, in economics, the shortest program is a list, carried out by a planner. But we know that the price system is a longer program but has greater explanatory power.

Short is not best, if ‘short’ means lacking explanatory power (why should this string be such and such).

Much of programming benefited from the idea of true/false binary implication coming from Whitehead and Russell. Because T/F tables translate into binary, but not qualitative logic.

The problem is that Whitehead and Russell’s ‘system of logic’ is not a system of logic at all. Its theory of non-necessary correspondence in two-value representation. Its basic assumption is logical positivism: empirical true-false implication of material propositions instead of distribution of middle terms.

This is the contra deduction epistemology.

Remember, Turing’s machine is a black box. The only time you can be sure of an answer is when the tape stops being modified by the black box; but this approach ignores causality: the black box cannot be opened. This may be true for natural phenomenon at some point, but it is not true for deductive logic.

Remember, Whitehead and Russell denied synthetic a priori.

Well, if you need some sequence of 0’s and 1’s to calculate the probability of the next digit being a 1 or a 0, that would make it a posteriori. The method relies on empirical/historical data to determine something. If a method could be used to obtain the likelihood before any observations were made, it would be a priori.

Law of Marginal Utility, First Law of Thermodynamics, Binomial and Multinomial theorem, do not rely on any unprovable assumptions.

The Minimum Description Length takes this into account. It establishes a tradeoff between the descriptive power of the model and its complexity.

Hmm, I think you’re oversimplifying. I don’t understand exactly what you are saying about Whitehead & Russell but it is not true that a Turing machine is necessarily a black box. You can write a program which provides successive approximations to an answer. Markov Chain Monte Carlo (MCMC) algorithms are a great example of this approach.

Clayton -

Exactly.

The universal distribution (as it is called) is independent of any particular sequence of 0’s and 1’s but to get a specific probability you must, of course, specify the sequence of interest.

The distribution itself can be constructed without reference to any particular observation.

I sharply beg to differ.

Clayton -

Citing the link Clayton posted: ‘We start by describing objects with binary strings. We take the length of the shortest string description of an object as a measure of that object’s complexity.’

This prevents full distribution of middle terms.

Why is that bad? For example,

empirical correspondence becomes logical identity in many instances, if a two value system is used. Recall Blanshard’s example: for all skies, sky is blue implies Chinese communists invaded Korea, etc., etc.

This is clearly meaningless, but that’s the basis of Principia Mathematica published in 1910 and many things that built on it.

Roger Penrose, more recently, showed this is precisely the reason why computers cannot check proofs that humans can see are true: true but not computable.

It is because a two-value system, instead of a formal infinite-value system (e.g. qualitative lattice of nonidentical nodes as in De Morgan-Jevons’ system), that as you’ve mentioned: many things become ‘true for no reason.’

I agree, however, that in many cases, instructions can be written in binary, which is as good as any quantitative system. Anything where number exists can be reproduced in binary. But there are logical propositions which are qualitatively complex, hence cannot be expressed in number form without loss of meaning. (Hence the derivation series: one quality system → many quantity systems → many geometries per quantity system.)

Probabilities can only describe measurable events. If an event is not measurable, then no probability can be assigned to it because it is impossible to distinguish between occurrence and non-occurrence of the event. Note that I am not saying that probability can only be assigned to objective events since I could, for example, construct a “probability that I am pleased by dinner at any given restaurant.” The variable is subjective but it is measurable. The device performing the measurement is my own subjective, conscious judgment of the pleasure of dinner out.

Clayton -

The universal distribution was not in the introduction you linked and I’m not familiar with AIT. Could you provide links which explain the theory we are discussing?

What’s the probability of getting a 1 at the end of an unspecified sequence of ones and zeros? And what is the AIT proof of this probability?

Show me.

But can you back up your difference?

I understand what you mean. I was not talking about measurable versus not-measurable events. I was talking about a system of logic itself. A system that is based in two-value ‘logic’ is not capable of deduction per se. Its only capable of empirical (or a priori correspondence) implication.

In the theory you suggested, virtually none of the implications are necessary; they are implied because of the Razor rule. If fact, to get logically necessary predicates, certain or probable, what we want is a program that may or may not be longer but capable of taking into meaningful account of all quantitative and qualitative material propositions that a person possesses such that he may make certain inference or probable inference. That means distribution of the middle terms of each proposition (which is excluded by the Razor rule as not important). Then the inference can be probable, but still necessary (only one of several logically necessary predicates).

Note: I was just talking about W & R’s 1910 PM because that’s the first two-value ‘alternative logic.’ For instance, since W & R never provided for logically necessary inference, only for correspondence of truth values, had to do without making any actual ‘formal logic.’ Godel’s original paper for incompleteness, in the last translation, is basically a verbal proof, because W & R’s two-value logic is not capable of proving anything as true because necessarily. He used two-value logic in smatterings, but did not use it to prove his case; only to repeat what he said.

Edit: Mises proved diminishing marginal utility is an apriori concept. It goes back to joining ordinal theory of Cuhel with Menger’s ‘wheat farmer.’ If I have ordinal preferences, then a quantity of one good is distributed among all the uses it satisfies, following the same diminishing scale of rankings. Also, for concrete partial satisfactions of any one preference. Hence, so long as ordinal preferences, then diminishing marginal utility. And Mises showed why action requires preference rankings. And action is synthetic a priori. Proof.