6.835  Intelligent Multimodal User Interfaces

Prerequisites: 6.034
6.170 will be useful but other programming experience is a satisfactory alternative. This course is distinct in subject matter from 6.831 (User Interface Design and Implementation) and is an appropriate follow-on.
 
Units: 3-0-9
Appropriate for an Engineering Concentration in AI and Applications.
Open to and appropriate for advanced undergraduates and graduate students.
 
This course will cover the design, implementation, and evaluation of intelligent and multi-modal user interfaces.  It will involve a combination of papers from the original literature and hands-on exercises to familiarize students with the basic technologies for handling speech, vision, and pen-based interaction, and enable them to explore architectures for combining modalities in a variety of ways.
 
There are substantial readings from original literature that go with each lecture. Students will have the opportunity to implement example interfaces in a series of course mini-projects, as well as a final project of their own design.
 
Topics will include: 
 
    • Gaze tracking and gaze-aware interfaces
    • Gesture recognition
    • Mobile image-based interfaces
    • Multimodal conversational models 
    • Audiovisual speech recognition
    • Pen-based interaction
    • Multimodal fusion and adaptation algorithms
    • Methods for combining channels and/or mutual disambiguation

Last thing to do for class: fill out the HKN survey: https://sixweb.mit.edu/student/evaluate/6.835-s2009