In science, maximum entropy principle, aka "MEP", MaxEnt principle, or “maximum entropy production principle” (MEPP) , as some refer to it, states that when an inference is made on the basis of incomplete information, it should be drawn from the probability distribution that maximizes the entropy (Shannon entropy) subject to the constraints on the distribution, and subsequently the maximum entropy probability distribution is the one which is maximally noncommittal with regard to the missing information, whereby the resulting maximum entropy probability distribution is supposed to have a maximum uncertainty or entropy associated with it. [1]
Overview
In 2015, English biosphere researcher Keith Skene, in his “Life’s a Gas: a Thermodynamic Theory of Biological Evolution”, reviewed by Umberto Lucia, attempted to connect the so called “maximum entropy production principle” (MEPP) to Marcellin Berthelot's heat theory of affinity, John Strutt’s dissipation function, Lars Onsager’s barrier reduction entropy model, and then to Hans Zeiglier, whom he credits with the formal definition of MEPP, and then to Edwin Jaynes’ Entropy Concentration Theory. [4]
The principle supposedly came from the 1957 work of American physicist Edwin Jaynes who utilized Shannon entropy in attempts to find connections to Clausius entropy. [2] This principle is associated with the MaxEnt school of thermodynamics.
The principle is now mostly used in computer science and information theory, having basically no connection to thermodynamics, albeit its name “maximum entropy principle” is often found mixed into uneducated attempts at applications of the second law to the study of physical systems, such as ecological systems, or evolution, etc. [3]
References
1. Karmeshu, Pal N.R. (2003). “Uncertainty, Entropy, and Maximum Entropy Principle: an Overview”, in: Entropy Measures, Maximum Entropy Principle and Emerging Applications (editor: Karmeschu (§1:1-54; pg. 31). Springer.
2. (a) Jaynes, E. T. (1957) “Information theory and statistical mechanics”, (PDF), Physical Review 106:620.
(b) Jaynes, E. T. (1957) “Information theory and statistical mechanics II”, (PDF), Physical Review 108:171.
3. Chakrabarti, C.G. and Ghosh, Koyel. (2009). “Maximum-entropy principle: Ecological Organization and Evolution (abs)”, Journal of Biological Physics, Aug. 03.
4. Skene, Keith R. (2015). “Life’s a Gas: a Thermodynamic Theory of Biological Evolution” (abs), Entropy, 17:5522-48.
Further reading
● Plischke, Michael and Bergersen, Birger. (1994). Equilibrium Statistical Physics (section 2.5: Maximum Entropy Principle, pgs. 48-51). World Scientific.
External links
● Principle of maximum entropy – Wikipedia.
● Maximum entropy principle – N. Sukumar, UC Davis.
