6.437 Inference and Information
Spring 2009
Reverend Thomas Bayes
Instructors: Polina Golland, Gregory W Wornell
TAs: Danial Lashkari, Maryam Modir Shanechi, Urs Niesen
Lecture:
Tuesdays and Thursdays, 9.30-11am
(2-105)
Recitations: Fridays, 10am or 11am
(26-322)
Office Hours: TAs: M 5-6, W 10:35-11:35 (32-044F); Golland: Thu 11-12 (32D-470); Wornell: T 11-12 (36-677)
Information:
Introduction to principles of Bayesian and non-Bayesian statistical inference. Hypothesis testing and parameter estimation, sufficient statistics; exponential families; EM algorithm. Log-loss inference criterion, entropy and model capacity. Kullback-Leibler distance and information geometry. Asymptotic analysis and large deviations theory. Model order estimation; nonparametric statistics. Computational issues and approximation techniques; Monte Carlo methods. Selected special topics such as universal prediction and compression.
ACCESS TO THE WEBSITE is limited to the
students enrolled in the course as LISTENERS or
FOR CREDIT. Please talk to us in class about
registering and getting access to the web site. You can download
the general information handout by clicking on the "General
Info" tab on the left, even if you are not enrolled in the
course.
Announcements
Final Week of the Term
Hi All--Just a reminder that our third quiz is this Thursday, May 14, 9-11am, on the top floor of Walker Memorial, as usual. It is again closed book, but you are permitted to bring three 8.5x11" sheets of notes (both sides) to the quiz.
The final revisions to the courses for the term have been uploaded to the web site. The most recent changes include uploading the Model Selection notes, and expanding the discussion and development of universal prediction in the Asymptotics notes. Note, too, that we have made a variety of corrections and clarifications to various portions the notes on the web site as the term has progressed (and after they were first uploaded), so be sure you have the latest versions.
cheers,
Greg
Announced on 11 May 2009 4:23 p.m. by Gregory W Wornell
Quiz 1 -- March 5, 9-11am, Walker Memorial
This is a quick reminder that our first quiz is next Thursday, March 5, 9:00-11:00am, on the top floor of Walker Memorial. You are allowed to bring one two-sided sheet of notes to the quiz.
Best,
Polina.
Announced on 27 February 2009 9:21 a.m. by Polina Golland
Inference Demonstration
Hi All--
Welcome to 6.437. In class today, you got some sense for the breadth of applications for the statistical inference framework we will develop over the term. To give you some sense of the power of this framework, we've added an interactive demo to the Materials section of the web site.
In this demo, you will play rock/paper/scissors against a javascript program. This program was written by a former student, Emin Martinian. We encourage you to try it out, and try to guess the principles on which it was designed. You will notice that it is hard to beat the program in the long run.
In fact, it is possible to show that the program will always eventually deduce your playing strategy, and then go on to win consistently, provided your approach requires only finite memory to implement. As the term unfolds, you'll gain a greater appreciation of this example.
Have fun!
Greg
Announced on 03 February 2009 3:43 p.m. by Gregory W Wornell
