Unit 4: Probability, Lecture 2

Flash and JavaScript are required for this feature.

Download the video from iTunes U or the Internet Archive.

This video is the second of two lectures in this unit.

Instructors: Prof. Paul Penfield, Prof. Seth Lloyd 

Probability is nothing but common sense reduced to calculation.
—Pierre-Simon Laplace (1749-1827)

Readings

Notes, Chapter 5: Probability (PDF)

Huffmann, David A. "A Method for the Construction of Minimum-Redundancy Codes." Proceedings of the Institute of Radio Engineers 40 (September 1952): 1098-1101.

English Letter Usage Statistics from A Tale of Two Cities by Charles Dickens

Towser's Wonderland Park greyhound handicaps, Boston Globe, February 27, 2005 (and results, Boston Globe, February 28, 2005)

Homework

Problem Set 4 (PDF)

Problem Set 4 Solutions (PDF)

Resources

Technical

Amazon logo Salomon, David. "Huffman Coding," and "Facsimile Compression using Huffman Coding." Section 2.8 and 2.13 in Data Compression. London, England: Springer, 2006. ISBN: 9781846286025.

The Human Mortality Database from University of California, Berkeley.

MIT current year student enrollment data: all students and women students

Sebastiani, Paola. "A Tutorial on Probability Theory." (PDF)# One of many good tutorials on the subject.

Historical

Amazon logo David, F. N. Games, Gods and Gambling. London, England: Charles Griffen & Co., 1962. Reprint, Dover 1998. ISBN: 9780486400235.

Girolamo Cardano (1501-1576), the first mathematician to calculate probabilities correctly.

Thomas Bayes (1702-1761)

David A. Huffman (1925-1999) obituary

Books

There are many excellent texts on probability, many of which do not assume a familiarity with mathematics beyond introductory calculus. Most books on communications include a summary of the necessary background in probability.

Amazon logo Drake, Alvin W. Fundamentals of Applied Probability Theory. New York, NY: McGraw-Hill, 1967. ISBN: 9780070178151.
Prof. Drake taught 6.041 Probabilistic Systems Analysis for many years before he retired. He died Oct. 30, 2005. (Obituary)

Amazon logo Bertsekas, Dimitri P., and John N. Tsitsiklis. Introduction to Probability. Belmont, MA: Athena Scientific, 2002. ISBN: 9781886529403.
Used in 6.041 today.

Amazon logo Applebaum, David. Probability and Information. New York, NY: Cambridge University Press, 2008. ISBN: 9780521727884.
Chapter 4 contains a good comparison of the different philosophies underlying probability (symmetry, subjective, frequency).

Amazon logo Haykin, Simon. Communication Systems. 4th ed. New York, NY: John Wiley and Sons, Inc., 2000. ISBN: 9780471178699. Appendix 1, Probability Theory.