In science, Claude Elwood Shannon (1916-2001) was an American electrical engineer and mathematician noted for his 1948 article “A Mathematical Theory of Communication”, in which, he situated a new type of communication "entropy" (symbol H), completely unrelated to thermodynamics, described as a measure of “information, choice and uncertainty”. In this paper, Shannon infamously states, in a subtle, yet very influential, way that seemed to connect his work to thermodynamics, that:


Then, after explaining his formulation of H as a function of a set of probabilities involved in the transmission of information (line currents), he concludes, in reference to Austrian physicist Ludwig Boltzmann's famous 1882 paper Further Studies on the Thermal Equilibrium of Gas Molecules, "H is then ... the H of Boltzmann's famous H theorem." [1] Boltzmann's H theorem, of course, was a statistical formulation of Clausius entropy (1865), for an ideal gas system. In short, what Shannon did, for whatever motive, was convolute telegraph theory together with heat engine theory as though it were the same subject matter, via a very crude unsubstantiated (verbalized) derivation. In 1961, to cite one opinion on this matter, French mathematician Benoit Mandelbrot commented that: [3]

“Everyone knows that Shannon’s derivation is in error.”

A great thermodynamicist?
One of the greatest misconceptions in modern science, perpetuated most likely from the fragmenting and relative isolation of the various modern branches of science, is the view that Shannon was a great thermodynamicist. This is a hugely erroneous myth. Shannon was an electrical engineer who studied the transmission and coding of voltage and electrical currents in telephone wires, no more no less. The connection he has to thermodynamics was counseling he received in the 1940s from Hungarian chemical engineer John Neumann on what name he should give to his new probability formula for the measure of information, uncertainty, or choice in telegraph wires. Shannon, however, never had any formal education in thermodynamics and his work had nothing to do with thermodynamics. Yet, for reasons which require further discussion, in online polls as to who is the greatest thermodynamicist of all-time?, Shannon’s name pops up, which is a puzzling phenomenon. [6]

Questionable application in other fields
In the years to follow, Shannon was at pains to contain the mess he had sowed. In 1955, for instance, Shannon had commented to the effect that one should not use his information theory in other than the analysis and study of current transmissions. In particular: [4]

“Workers in other fields should realize that the basic results of the subject [communication channels] are aimed in a very specific direction, a direction that is not necessarily relevant to such fields as psychology, economics, and other social sciences.”

Likewise, after hearing of Mandelbrot’s 1961 criticism, Shannon continued to express “misgivings about using his definition of entropy for applications beyond communication channels.” [3] In any event, Shannon's warnings didn't help and in the decades to follow his euphemistic verbalized derivation of his communication theory H function in relation to Boltzmann’s statistical thermodynamic H function soon led hundreds of individuals (examples: James Coleman (1964), Stephen Coleman (1975), Orrin Klapp (1978), Jay Teachman (1980), Kenneth Bailey (1990), etc.) to write theoretical papers and books on connections between communication, information, entropy, and thermodynamics; all of which, of course, being unsubstantiated. Shannon's formulation has come to be known as information entropy, Shannon entropy, information theoretic entropy, among others.

Tribus
To cite one dominant example of the influence of Shannon's thermodynamics-borrowed terminology, in 1948 American engineer Myron Tribus was asked during his examination for his doctoral degree, at UCLA, to explain the connection between Shannon entropy and Clausius entropy. In retrospect, in 1998, Tribus commented that he went on to spend ten-years on this issue: [3]

“Neither I nor my committee knew the answer. I was not at all satisfied with the answer I gave. That was in 1948 and I continued to fret about it for the next ten years. I read everything I could find that promised to explain the connection between the entropy of Clausius and the entropy of Shannon. I got nowhere. I felt in my bones there had to be a connection; I couldn’t see it.”

Information entropy

See main: Information entropy, Shannon entropy, etc.

Shannon’s revolutionary idea of digital representation was to sample the information source at an appropriate rate, and convert the samples to a bit stream. He characterized the source by a single number, the entropy, adapting a term from statistical mechanics, to quantify the information content of the source. For English language text, Shannon viewed entropy as a statistical parameter that measured how much information is produced on the average by each letter. [2] The equation for H that Shannon defines as entropy is:

in which H is a measure of "information, choice and uncertainty", the constant K is a variable that "merely amounts to a unit of measurement", and p1, ..., pn are a "set of probabilities." Through this equation, dozens of writers have, unknowingly, jumped from verbal descriptions of all varieties of information, e.g. genetic, computer, knowledge, etc., to mixtures of Boltzmann-Clausius statements of the second law of thermodynamics, in attempts to explain enumerable aspects of life and evolution, among other subjects.

Difficulties on theory
The essential difficulty of Shannon’s idea of entropy is that it’s terminology is a verbal-crossover theory, culled from statistical thermodynamics, but having little connection, if any at all, to thermodynamics. This has lead countless writers, having little training in thermodynamics, to proffer up some of the most illogical backwardly reasoned papers every written, however novel the intentions. These papers, from a thermodynamic point of view, become an almost strain on the mind to read. The 2007 views of German physicist Ingo Müller summarizes the matter near to a tee: [3]

“No doubt Shannon and von Neumann thought that this was a funny joke, but it is not, it merely exposes Shannon and von Neumann as intellectual snobs. Indeed, it may sound philistine, but a scientist must be clear, as clear as he can be, and avoid wanton obfuscation at all cost. And if von Neumann had a problem with entropy, he had no right to compound that problem for others, students and teachers alike, by suggesting that entropy had anything to do with information.”

Müller clarifies that “for level-headed physicists, entropy (or order and disorder) is nothing by itself. It has to be seen and discussed in conjunction with temperature and heat, and energy and work. And, if there is to be an extrapolation of entropy to a foreign field, it must be accompanied by the appropriate extrapolations of temperature, heat, and work.”

Education
Shannon completed a BS in electrical engineering and a BS in mathematics at the University of Michigan in 1936. He completed a MS, thesis A Symbolic Analysis of Relay and Switching Circuits, 1937, and PhD, thesis An Algebra for Theoretical Genetics, 1940, at the Massachusetts Institute of Technology.

References
1. Shannon, Claude E. (1948). "A Mathematical Theory of Communication", Bell System Technical Journal, vol. 27, pp. 379-423, 623-656, July, October.
2. Claude Shannon 1916-2001 – Research.ATT.com.
3. Muller, Ingo. (2007). A History of Thermodynamics - the Doctrine of Energy and Entropy (ch 4: Entropy as S = k ln W, pgs: 123-126). New York: Springer.
4. Tribus, M. (1998). “A Tribute to Edwin T. Jaynes”. In Maximum Entropy and Bayesian Methods, Garching, Germany 1998: Proceedings of the 18th International Workshop on Maximum Entropy and Bayesian Methods of Statistical Analysis (pgs. 11-20) by Wolfgang von der Linde, Volker Dose, Rainer Fischer, and Roland Preuss. 1999. Springer.
5. (a) IEEE Transactions on Information Theory, December 1955.
(b) Hillman, Chris. (2001). “Entropy in the Humanities and Social Sciences.” Hamburg University.
6. Orzel, Chad. (2009). “Historical Physicist Smackdown: Thermodynamics Edition”, Science Blogs, Uncertainty Principles.

Further reading
● Shannon, Cluade E. (1950), "Prediction and Entropy of Printed English", Bell Sys. Tech. J (3) p. 50-64.