6. Knowledge, Information, and Entropy
The book John von Neumann and the Foundations of Quantum
Physics contains a fascinating and informative article written by
Eckehart Kohler entitled “Why von Neumann Rejected Carnap’s
Dualism of Information Concept.” The topic is precisely the core issue
before us: How is knowledge connected to physics? Kohler
illuminates von Neumann’s views on this subject by contrasting them
to those of Carnap.
Rudolph Carnap was a distinguished philosopher, and member of the
Vienna Circle. He was in some sense a dualist. He had studied one
of the central problems of philosophy, namely the distinction between
analytic statements and synthetic statements. (The former are true or
false by virtue of a specified set of rules held in our minds, whereas
the latter are true or false by virtue their concordance with physical or
empirical facts.) His conclusions had led him to the idea that there are
two different domains of truth, one pertaining to logic and
mathematics and the other to physics and the natural sciences. This
led to the claim that there are “Two Concepts of Probability,” one
logical the other physical. That conclusion was in line with the fact
that philosophers were then divided between two main schools as to
whether probability should be understood in terms of abstract
idealizations or physical sequences of outcomes of measurements.
Carnap’s bifurcations implied a similar division between two different
concepts of information, and of entropy.
In 1952 Carnap was working at the Institute for Advanced Study in
Princeton and about to publish a work on his dualistic theory of
information, according to which epistemological concepts like
information should be treated separately from physics. Von
Neumann, in private discussion, raised objections, and Pauli later
wrote a forceful letter, asserting that “I am quite strongly opposed to
the position you take.” Later he adds “I am indeed concerned that the
confusion in the area of the foundations of statistical mechanics not
grow further (and I fear very much that a publication of your work in
its present form would have this effect).”
Carnap’s view was in line with the Cartesian separation between a
domain of real objective physical facts and a domain of ideas and
concepts. But von Neumann’s view, and also Pauli’s, linked the
probability that occurred in physics, in connection with entropy, to
knowledge, in direct opposition to Carnap’s view that epistemology
(considerations pertaining to knowledge) should be separated from
physics. The opposition of von Neumann and Pauli significantly
influenced the publication of Carnap’s book.
This issue of the relationship of knowledge to physics is the central
question before us, and is in fact the core problem of all philosophy
and science. In the earlier chapters I relied upon the basic insight of
the founders of quantum theory, and upon the character of quantum
theory as it is used in actual practice, to justify the key postulate that
Process I is associated with knowing, or feeling. But there is also an
entirely different line of justification of that connection developed in
von Neumann’s book, Mathematical Foundations of Quantum
Mechanics. This consideration, which strongly influenced his thinking
for the remainder of his life, pertains to the second law of
thermodynamics, which is the assertion that entropy (disorder,
defined in a precise way) never decreases.
There are huge differences in the quantum and classical workings of
the second law. Von Neumann’s book discusses in detail the
quantum case, and some of those differences. In one sense there is
no nontrivial objective second law in classical physics: a classical
state is supposed to be objectively well defined, and hence it always
has probability one. Consequently, the entropy is zero at the outset
and remains so forevermore. Normally, however, one adopts some
rule of “coarse graining” that destroys information and hence allows
probabilities to be different from unity, and then embarks upon an
endeavor to deduce the laws of thermodynamics from statistical
considerations. Of course, it can be objected that the subjective act of
choosing some particular coarse graining renders the treatment not
completely objective, but that limited subjective input seems
insufficient to warrant the claim that physical probability is closely tied
to knowledge.
The question of the connection of entropy to the knowledge and
actions of an intelligent being was, however, raised in a more incisive
form by Maxwell, who imagined a tiny “demon” to be stationed at a
small doorway between two large rooms filled with gas. If this agent
could distinguish different species of gas molecules, or their energies
and locations, and slide a frictionless door open or closed according
to which type of molecule was about to pass, he could easily cause a
decrease in entropy that could be used to do work, and hence to
power a perpetual motion machine, in violation of the second law.
This paradox was examined Leo Szilard, who replaced Maxwell’s
intelligent “demon” by a simple idealized (classical) physical
mechanism that consumed no energy beyond the apparent minimum
needed to ‘recognize and responded differently to’ a two-valued
property of the gas molecule. He found that this rudimentary process
of merely ‘coming to know and respond to’ the two-valued property
transferred entropy from heat baths to the gaseous system in just the
amount needed to preserve the second law. Evidently nature is
arranged so that what we conceive to be the purely intellectual
process of coming to know something, and acting on the basis of that
knowledge, is closely linked to the probabilities that enter into the
constraints upon physical processes associated with entropy.
Von Neumann describes a version of this idealized experiment.
Suppose a single molecule is contained in a volume V. Suppose an
agent comes to know whether the molecule lies to the left or to the
right of the center line. He is then in the state of being able to order
the placement of a partition/piston at that line and to switch a lever
either to the right or to the left, which restricts the direction in which
the piston can move. This causes the molecule to drive the piston
slowly to the right or to the left, and transfer some of its thermal
energy to it. If the system is in a heat bath then this process extracts
from the heat bath an amount ‘log 2’ of entropy (in natural units).
Thus the knowledge of which half of the volume the molecule was in
is converted into a decrement of “log 2’ units of entropy. In von
Neumann’s words, “we have exchanged our knowledge for the
entropy decrease k log 2.” (k is the natural unit of entropy.)
What this means is this: When we conceive of an increase in the
“knowledge possessed by some agent” we must not imagine that this
knowledge exists in some ethereal kingdom, apart from its physical
representation in the body of the agent. Von Neumann’s analysis
shows that the change in knowledge represented by Process I is
quantitatively tied to the probabilities associated with entropy.
Among the many things shown by von Neumann are these two:
(1) The entropy of a system is unaltered when the state of that
system is evolving solely under the governance of Process II.
(2) The entropy of a system is never decreased by any Process I
event.
The first result is analogous to the classical result that if an objective
“probability” were to be assigned to each if a countable set of
possible classical states, and the system were allowed to evolve in
accordance with the classical laws of motion then the entropy of that
system would remain fixed.
The second result is a nontrivial quantum second law of
thermodynamics. Instead of coarse graining one has Process I, which
in the simple ‘Yes-No’ case converts the prior system into one where
the question associated with the projection operator P has a definite
answer, but only the probability associated with each possible answer
is specified, not an answer itself.
One sees, therefore, why von Neumann rejected Carnap’s attempt to
divorce knowledge from physics: large tracts in his book were
devoted to establishing their marriage. That work demonstrates the
quantitative link between the increment of knowledge or information
associated with a Process I event and the probabilities connected to
entropy. This focus on Process I allowed him to formulate and prove
a quantum version of the second law. In the quantum universe the
rate of increase of entropy would be determined not by some
imaginary and arbitrary coarse graining rule, but by the number and
nature of objectively real Process I events.
Kohler discusses another outstanding problem: the nature of
mathematics. At one time mathematics was imagined to be an
abstract resident of some immaterial Platonic realm, independent in
principle from the brains and activities of those who do it. But many
mathematicians and philosophers now believe that the process of
doing mathematics rests in the end on mathematical intuitions, which
are essentially aesthetic evaluations.
Kohler argues that von Neumann held this view. But what is the origin
or source of such aesthetic judgments?
Roger Penrose based his theory of consciousness on the idea that
mathematical insight comes from a Platonic realm. But according to
the present account each such illumination, like any other experience,
is represented in the quantum description of nature as a picking out
of an organized state in which diverse brain processes act together in
an harmonious state of mutual support. A mathematical illumination
is a grasping of an aesthetic quality of order in the quantum state of
the agent’s brain/body. But apparently every experience of any kind is
fundamentally like this: it is a Process I grasping of a state of order.
This notion that each Process I event is a felt grasping of a state in
which various sub-processes act in concert provides a foundation for
answering in a uniform way many outstanding philosophical and
scientific problems. For example, it provides a foundation for a
solution to a basic issue of neuroscience, the so-called “binding
problem”. It is known that diverse features of a visual scene, such as
color, location, size, shape, etc. are processed by separate modules
located in different regions of the brain. This understanding of the
Process I event makes the felt experience a grasping of a non-
discordant quasi-stable mutually supportive combination of these
diverse elements as a unified whole. To achieve maximal
organizational impact this event should provide the conditions for a
rapid sequence of re-enactments of itself. Then this conception of the
operation of von Neumann’s process I provides also an
understanding of the capacity of an agent’s thoughts to control its
bodily behavior. The same conception of Process I provides also a
basis for understanding both artistic and mathematical creativity, and
the evolution of consciousness in step with the biological evolution of
our species. These issues all come down to the problem of the
connection of knowings to physics, which von Neumann’s treatment
of entropy ties to Process I.