Comment by sixo

This comment thread is exhibit N-thousand that "nobody really understands entropy". My basic understanding goes like this:

In thermodynamics, you describe a system with a massive number of microstates/dynamical variable according to 2-3 measurable macrostate variables. (E.g. `N, V, E` for an ideal gas.)

If you work out the dynamics of those macrostate variables, you will find that (to first order, i.e. in the thermodynamic limit) they depend only on the form of the entropy function of the system `S(E, N, V)`, e.g. Maxwell relations.

If you measured a few more macrostate variables, e.g. the variance in energy `sigma^2(E)` and the center of mass `m`, or anything else, you would be able to write new dynamical relations that depend on a new "entropy" `S(E, N, V, sigma^2(E), m)`. You could add 1000 more variables, or a million—e.g every pixel of an image—basically up until the point where the thermodynamic limit assumptions cease to hold.

The `S` function you'd get will capture the contribution of every-variable-you're-marginalizing-over to the relationships between the remaining variables. This is the sense in which it represents "imperfect knowledge". Entropy dependence arises mathematically in the relationships between macrostate variables—they can only couple to each by way of this function which summarizes all the variables you don't know/aren't measuring/aren't specifying.

That this works is rather surprising! It depends on some assumptions which I cannot remember (on convexity and factorizeabiltiy and things like that), but which apply to most or maybe all equilibrium thermodynamic-scale systems.

For the ideal gas, say, the classical-mechanics, classical-probability, and quantum-mechanic descriptions of the system all reduce to the same `S(N, V, E)` function under this enormous marginalization—the most "zoomed-out" view of their underlying manifold structures turns out to be identical, which is why they all describe the same thing. (It is surprising that seemingly obvious things like the size of the particles would not matter. It turns out that the asymptotic dynamics depend only on the information theory of the available "slots" that energy can go into.)

All of this appears as an artifact of the limiting procedure in the thermodynamic limit, but it may be the case that it's more "real" than this—some hard-to-characterize quantum decoherence may lead to this being not only true in an extraordinarily sharp first-order limit, but actually physically true. I haven't kept up with the field.

No idea how to apply this to gravity though.