Unit 7: Inference, Lecture 1

Flash and JavaScript are required for this feature.

Download the video from iTunes U or the Internet Archive.

Resources

Technical

The person most responsible for use of maximum entropy principles in various fields of science is Edwin T. Jaynes. The seminal papers are:

Jaynes, E. T. "Information Theory and Statistical Mechanics." Physical Review 106 (May 15, 1957): 620-630. (PDF - 2.1 MB)# (PS - 2.5 MB)
This paper started the modern use of the Principle of Maximum Entropy in physics.

———. "Information Theory and Statistical Mechanics II." Physical Review 108 (October 15, 1957): 171-190. (PDF - 3.6 MB)# (PS - 4.5 MB)
Continuation of the previous reference.

Historical

Edwin T. Jaynes (1922-1998) photograph, biography, bibliography, list of unpublished works, and book reviews, from G. Larry Bretthorst

Jaynes knew, of course, about Thomas Bayes and when on sabbatical in England sought out and photographed the Bayes tombstone.

Books

Tribus, M. Thermostatics and Thermodynamics. Princeton, NJ: D. Van Nostrand Co, Inc., 1961.
An early textbook to use the Principle of Maximum Entropy as an approach to thermodynamics. The philosophy of assuming maximum uncertainty is discussed in Chapter 3.

Jaynes, E. T. "Information Theory and Statistical Mechanics." In Statistical Physics. 1962 Brandeis Lectures in Theoretical Physics, Volume 3. New York, NY: W. A. Benjamin, Inc., 1963. pp. 181-218. (PDF)# (PS - 1.1 MB)
Another good explanation, in terms of estimating probabilities of an unfair die.

Amazon logo Personal history by Jaynes, Edwin T. "Where Do We Stand on Maximum Entropy?" In The Maximum Entropy Formalism. Edited by Raphael D. Levine and Myron Tribus. Cambridge, MA: MIT Press, 1978. pp. 15-118. ISBN: 9780262120807. (PDF - 2.5 MB)# (PS - 3.2 MB)