Entropy and Knightian Uncertainty, a subtle connection?

Economics Economics Questions
1

I’ve just hit upon a speculation that I think might be worth exploring later, and thought others might find it interesting if i shared it. To inform the argument, consider the following 2 statements, one of them a quote from Human Action, the other, a statement of the 2nd Law of Thermodynamics.

“The notion of change implies the notion of temporal sequence.”

pg 99, Human Action.

“As time progresses, the entropy of a closed system cannot decrease, but can only stay the same or increase, with a tendency to increase as time tends to infinity (largely rooted in the general empirical observation that the universe we are part of does not consist entirely of reversible thermodynamic equilibrium systems).”

My schoolboy statement of the 2nd Law of Thermodynamics.

Strangely enough, I find these statements (as well as Mises statements later in the probabillity chapter) to complement each other quite well. The universe is one of genuine change, and hence does not consist of simplistic toy model mechanical systems which are entirely cyclical and regular, given the 2nd law which crtitically is the only physical which is time asymmetric. It is the 2nd law which objectively helps us establish a direction of time. Even though the law has been experimentally confirmed (it is along with the other laws of thermodynamics, possibly the most certain statement of physical law available to us), it is interesting that we could not even consistently comprehend the nature of action in a universe without it, without running into contradictions as Mises later elaborates.

Yet this would offend and rule out any attempt to model literally equilibrium constructions, which repeat themselves lifelessly, adding on top of the objections(or rather as an integrated complement) made by Mises from a legitmate epistemological viewpoint, in the sense that they are contrary to the very nature and reasoning behind action; that they would also contradict the 2nd law of thermodynamics.

Irealise this might be starting to sound a little nutty however, since conventionally, thermodynamics analysis is restricted to entirely “physical” contexts, yet I think an understanding of information and chaos theory as developed in the 20th century may help us in finding the inter-relation between these concepts.

Chaos theory as first developed out of Poincare’s attempt to solve the 3 body problem, damaged the overtly simplistic determinism of classical mechanics, by demonstrating the divergence which must occur for different solutions of a system, given slight alterations of their initial conditions (and hence by implication, our knowledge of them), placing an epistemic limit on human prediction given the physical limits placed on the experimental apparatus available to human beings. (This is without even considering the additional caveats brought about by quantum mechanics)

Along a different line of thought, we have information theory, which took the second law and allowed it to be understood in a different context, yet also allowed the scope of its application to be broadened, with the transfer of signals and applications in dealing with data loss along communication channels and efficient memory allocation. Information theory tells us that entropy can be considered not only as a vague measure of “disorder,” but gives us a quantitative measure of the lack of information we have concerning the particularities of the structure and constituents of a physical system. And hence the 2nd Law can be stated in this context, saying that the information we have concerning the structure and constituents of a system must tend to decrease with time increasing.

I wrote an introduction to information theory last year that people may find useful, to help get to grips with this:

http://www.filefactory.com/file/b4e1db1/n/information-theory.pdf

Hence combining our understanding of the limits placed by the two notions of chaos and “information entropy” described above, information we can know as human beings about the evolution of all systems is severely limited on epistemic grounds, and outside of idealised environments, in which it can and has been established that external conditions have been isolated and held constant(e.g. with isolated and repeated experiments), uncertainty of a nonquantifiable form must exist, at least from the perspective of human beings. This is regardless of whether an actual probabillity distribution concerning all future events could be established from a “God’s eye view.” This is due to the chaotic nature of phenomena of which information can be established in a probabilistic sense. It goes without saying it is far from trivial how much of the phenomena encompassing our world, especially in the sphere of action could be described in such a way, it may hardly be at all, again given the inherent problem of what can be established as a repeated case with conditions kept constant, and hence treated as a composition of a class.

I guess maybe what I’m trying to say is that, there may even be a “case” nature to many phenomena that are considered class probabillities, in light of the above discussion.

So what do you think of this? Have I finally lost my mind?! Could this be a productive line of enquiry to further explore? Or is this muddled claptrap?

2

The second law is probablistic. If the system is free to have n number of isenthalpic states, it will tend to appear random, because there are grossly more apparently random states than apparently ordered ones. For example, if I throw a bunch of marbles on a pool table with a lot of force, they could stop anywhere. It is probable that they will appear randomly distributed, but it is possible that they will spell out my name.

From this, I don’t think the second law can show anything a priori. Its just when you talk about quantum states, or even just atoms, the number of parameters is so high that the chances of any particular state go to zero, but never reach it.

Moreover, the distinction made by the second law between ordered and disordered is subjective to the human experience. What is meant is that the probability of state X is 1/n, and n–>infinity, so P(X) ~ 0

Sorry… I don’t understand how this references the second law. Its all true… there are probably things out there for which our probability estimates are wildly off… particularly our estimation of events with low probabilitys are very very uncertain.

3

Yes of course, S=k ln(w), via Boltzmann’s definition whereby w represents the number of microstates(e.g. No of ways of arranging the marbles) the system can occupy for any given macrostate. Conventionally, we assume a Gaussian distribution(or binomial if the parameters are discrete) making the a priori assumption whose results are empirically attested that the individual microstates(including degenerate states counted individually), have equal a probabiilities of being “the state” of the system, you could say p=1/w. However, this also conveys an indeterminancy of the information we have of the system, from which we can come up with a probabillistic definition of information, as outlined in the pdf I linked. On top of that not all systems have Gaussian distributions, e.g. the alphabet, which becomes very important when designing communication channels with which to transmit information without losses, and to allocate bits of memory to each letter, and the link between both spheres is to do with the fact there is an entropy cost associated with finding out information of physical systems, hence there are also entropic losses that need to be accounted for in data transmission. Hence, Shannon entropy generalises the definition of entropy to all probabillity distributions whereby S=k p ln(p).

From the above of course, it’s not obvious why the 2nd law should have anything to do with critiquing the equilibrium constructions used in economics, yet though there are important differences between the constructions used in either field, they seem to assume that which we know is not possible and not general for almost all physical systems barring very special conditions, which is complete predictable time symmetric regularity of operation, without change as epitomised by most clearly by Mises when he critiques the idea of using ERE, as any more than a tool to understand the source of profit in human uncertainty about future conditions. Indeed it is a further irony that the general equilibrium approach was inspired by thermodynamics, though more on this could be said at a different time.

The 2nd law may not be a priori, but it helps us appreciate that closed and global systems generally cannot be perfectly regular and time symmetric, and we cannot even consistently conceive of such a state of affairs anyway, as Mises points out. Since this symmetry cannot even be a characteristic of almost all physical systems, then the burden of proof may really be with the static equilibrium theorists to establish how they can apply and assume the conditions they do to human systems in the real world. I guess in a way I’m kind of saying there might be more than one road that helps us get to Rome understanding these insights.

As far as I was aware it had not been established that the sheer number of parameters was the reason for the probabilistic nature of quantum mechanics(which to be honest, I supect has nothing to do with it, given that it is not extraordinary for a lot of systems), which I believe is an unsolved or even in principle unknowable problem since from quantum mechanics all we can know about a system can be encoded within its wavefunction (more accurately its statevector), which if we take the modulus squared over a spatial distribution(for example), we produce a probability distribution for where a single particle will lie in single particle quantum mechanics. The values the parameters can take are discrete though the extent of this depends on their momentum and this is expressed via Heisenberg’s uncertainty principle, and of course there are probabilities associated with the statevector being any one of the eigenvectors of the system(in a way similiar to the way in which a macrostate can have any given microstate oreintation, though the wavefunction cannot be broken down into the analysis of billiard balls that move entirely deterministically, and simply have a probabilistic distribution due to our lack of knowledge of initial conditions. In QM we can;t go any deeper than the probability distribution, it’s an ultimate given).

Sorry if that was the king of digressions, indeed I think it may well be I’m off my rocker with this stuff, it’s just that I see a lot of inter-relations between these different branches of knowledge sometimes. I find it strange how positivists think praxeology is at odds with the natural sciences, to me it seems to be the only branch of economics representing the kind of rigour and consistency found in the former.

Yes of course, and this is an epistemic problem which I do not think is trivial in a hell of a lot of cases, especially with regard to human events since one has to establish whether and to what extent things can be considered part of the same class, whether we can assume et ceteris paribus etc and this is especially so for systems whereby all we have is a certain sample from which we find it nontrivial to even establish the bounds of different classes(since this is done with reference to known theory from other branches in the physical sciences) and to make probabillity estimates.

I guess the original and confusing part is my conjecture combining the insights of information theory or thermodynamics with chaos theory. We know that almost all actual systems are nonlinear as opposed to linear, and a great many of these are chaotic, from which we can deduce very little in advance with certainty due to the divergences that can occur with slight changes in initial conditions, which we can only know in the first place to a limited degree of accuracy. What I am suggesting is that these may affect systems characterised by probabilistic or incomplete information about the characteristics of parameters, from which we may even necessarily see a shift even if “in principle” we can establish initial conditions with objective accuracy. I’m not sure if this conjecture has even been made and applied in the natural sciences to be honest.

But it adds an interesting problem on top of the others already raised by the Austrians, to those who would try to utilise complete and perfectly known pronbability distributions to model future human actions as if they were engaged in by actual human beings in the economic sphere. Indeed, a researh idea, I have hit upon is to use the very neoclassical apparatus against them by showing how when one uses the same mathematics they use, but allow for diveregences of expectations which may not actually match at all the true probability distribution characterising events in your “dummy system”, that this can help illustrate and simulate a lot of results the Austrians predict, and show the equilibrium guys how sterile their systems really are.

But more on that when I’ve composed my thoughts on this a little more. Any further comments are most welcome, after all this is speculation. I’ve finished my degree now, and received a 2:1, though if I got a first, I would have been tempted to investigate this very interesting stuff in physics, let alone anything else, but I’m not sure I’ve shown the competency for this kind of research yet. I still need to improve my mathematical skills…

4

Okay. Sieben is ignorant, probably. In the following post, I preemptively admit that I may miss many boats entirely:

I thought we assumed a flat distribution? States are not clustered around a mean ‘State’, but are all equally probable.

I don’t know what this means

My interpretation was that the very large numbers of parameters render otherwise unknowable outcomes (due to randomness) knowable because P(states with property X) → 1. The probabilistic nature of quantum mechanics wouldn’t have anything to do with the number of variables, but it obviously helps in predicting outcomes.

Why do they have to be discrete? I thought the space was continuous.

See I would almost say the opposite: that even though the world’s elements are wildly variable, the sheer number of them makes some overall properties of the system knowable. But what’s interesting about economic elements is that they are not independent, whereas in the billiard ball example we assume the positions are independent. So if element X can cause Y, the philosophy of entropy no longer applies. You would have to redefine the system to only consider independent elements… which would be kind of boring from a social sciences standpoint.

Little lost… like, changes in consumer preferences? Time preference?

I am regularly embarrassed by this.

5

Don’t worry about that, for the fact you’ve at least been engaging me, I more than appreciate your input, but your last 2 posts have also helped me realise how convoluted some of my statements have been and how haphazard some of my thinking has been, in the sense I have just been spitting out results, while forgetting how they were obtained in the first place, with your point above about distributions being especially poinient.

I was going to finish typing up my response to your post, but it’s getting long and it’s really late(or early) and I should get some sleep.

Please stay tuned, I hope to come back tomorrow(or the day after at the latest) to address your points. You know, all this came from reading a sentence of Human Action, then stopping and thinking about it! I need help…

In the mean time, somebody else might even be able to fill in some of the glaring gaps I’ve left for now. Good night, or morning…

6

I’m sorry about taking so long to reply. I think I may have been wrong or misleadingly stated a few things above. I could post what I’ve written up so far, but it wouldn’t really do justice. I wanted to consult my statistical physics notes, but never got round to going through them. I’ll try to come back to this thread, but I think it will be a while later.