Calendar

SES # TOPICS KEY DATES
1

Fully- vs. under-actuated systems

Preliminaries

  
2 Nonlinear dynamics of the simple pendulum Problem set 1 out
3

Introduction to optimal control

Double-integrator examples

  
4

Double integrator (cont.)

Quadratic regulator (Hamilton-Jacobi-Bellman (HJB) sufficiency), min-time control (Pontryagin)

  
5 Dynamic programming and value interation: grid world, double integrator, and pendulum examples

Problem set 1 due

Problem set 2 out
6 Acrobot and cart-pole: controllability, partial feedback linearization (PFL), and energy shaping  
7 Acrobot and cart-pole (cont.)  
8 Policy search: open-loop optimal control, direct methods, and indirect methods Problem set 2 due
9 Policy search (cont.): trajectory stabilization, iterative linear quadratic regulator (iLQR), differential dynamic programming (DDP) Problem set 3 out
10 Simple walking models: rimless wheel, compass gait, kneed compass gait  
11 Feedback control for simple walking models  
12 Simple running models: spring-loaded inverted pendulum (SLIP), Raibert hoppers  
  Midterm  
13 Motion planning: Dijkstra's, A-star Problem set 3 due
14 Randomized motion planning: rapidly-exploring randomized trees and probabilistic road maps Problem set 4 out
15 Feedback motion planning: planning with funnels, linear quadratic regulator (LQR) trees  
16 Function approximation and system identification Final project proposal due
17 Model systems with uncertainty: state distribution dynamics and state estimation Problem set 4 due
18 Stochastic optimal control Problem set 5 out
19 Aircraft  
20 Swimming and flapping flight Problem set 5 due
21 Randomized policy gradient  
22 Randomized policy gradient (cont.)  
23 Model-free value methods: temporal difference learning and Q-learning  
24

Actor-critic methods

Final project presentations

  
25 Final project presentations Final projects due