CURRENT: 12.S592 U/G MIT REGISTRATION Lec: F. 9-11 Rm: Zoom & 54-1827
Simultaneously offered at Instituto Politecnico National (IPN) Queretaro (via Zoom).
Instructor: Sai Ravela (ravela@mit.edu)
- Offered since 2017, in earlier form since 2013
- Dynamics, Optimization, and Information foundations of Machine Learning
- ML-based System Dynamics and Optimization for Earth, Planets, Climate, and Life
- Flexible participation model
- Pre-reqs: Linear Algebra, Probability & Stat, a first course in ML, and some Systems preparation (at least one of) Statistical Signal Processing, Estimation, Control or Optimization
Seminar course 12.S592 is offered every term. Its syllabus follows four primary themes annually with an almost periodic, ahem, “50-shades of challenging grey matter” over approximately 50 weeks. Some have stayed on in the course for six years!
Theme By Season
-
Traditional notions of Learning are centered around Generalization and Learnability, but both struggle as tools to improve the learning process. We begin with a few dilemmas:
- Universal Approximation vs. the Model Error disaster. Is ML the savior we’ve thought it would be? For what?
- No free lunch theorem
- Bias vs. Variance
- Invariance vs. Selectivity
- Generalization vs. Stability
- Predictability vs. Learnability
- Learning from Theory vs. Data
Then, we will frame the learning Loop — how might you design your learning experiments
- Cross-Validation, Bootstrap, Jacknife, Leave-one-out
- Regularization
- Data, Model, and Parameter Selection