Course Meeting Times
Lectures: 2 sessions / week, 1.5 hours / session
Course Description
This course will cover a collection of geometric techniques that apply broadly in modern algorithm design. The exact topics covered will depend on student interest, but a (perhaps overly ambitious) set of possibilities includes:
Spectral Graph Theory
Graph Laplacians and their eigenvalues, connections to random walks and mixing, isoperimetric and Cheeger inequalities, expanders, and random graphs. Applications to include graph cutting, clustering, approximate counting, disjoint path problems, routing, and graph drawing.
Convex Geometry
Geometric properties of high-dimensional convex bodies, Fritz John's theorem and isotropy, Brunn-Minkowski and isoperimetric inequalities, concentration of measure and connections to probability theory. Applications to include volume computation and convex programming.
Multiplicative Weights
The multiplicative weights update method, its geometric meaning, and the many ways that it appears in modern computer science, with a focus on its use in optimization. Applications to include fast approximation algorithms for graph problems, "boosting" in learning and complexity theory, online algorithms, and zero-sum games.
Iterative Methods for Linear Algebra
How to use geometric information to quickly solve linear systems and eigenvalue problems. Will cover basic iterative methods, the Lanczos algorithm, conjugate gradients, preconditioning, and how spectral graph theory can be used to improve the construction of preconditioners.
Lattices and Basis Reduction
Basic properties of lattices, Minkowski's theorem, and the LLL algorithm. Applications to include solving low-dimensional integer programs and breaking cryptosystems.
LP- and SDP-based Approximation Algorithms for NP-Hard Problems
Linear and semidefinite programming relaxations of NP-hard problems, rounding techniques, and primal-dual methods.
Course Goals
- To learn a collection of powerful (and interrelated) mathematical techniques for algorithm design
- Topics picked to be both mathematically interesting and practically useful
- Will sometimes have long mathematical interludes, but always with proportional algorithmic payoffs
- To be able to apply these tools directly to your research
- Whether your work is theoretical or applied
- I'll suggest open research questions whenever possible
Course Requirements
The course requirements will comprise two parts:
- Problem sets, and
- Scribe notes
1. Problem Sets
I do not want this course to present an onerous workload, but I do want to give out enough problems to allow people to really learn the material. Since this is an advanced graduate class, I believe that students are capable of deciding for themselves how best to make this tradeoff. As such, I will hand out problem sets for each major topic and correct them, but I will not expect you to do all of the problems that I distribute. I will expect students to do as many as is necessary for a good-faith effort to learn the material. I recommend a total equivalent to at least two graduate problem sets over the course of the semester. Undergraduates taking this class will have a slightly more concrete set of requirements in which the number of required problems will be specified more precisely.
2. Scribe Notes
I would like to have scribes for every lectures. Each student should do his/her fair share of these.
Collaboration Policy
Collaboration is encouraged on the problem sets. However, you should think about the problems yourself before discussing them with others. Furthermore, you must write your solutions up individually and understand anything that you hand in. If you do collaborate, you must acknowledge your collaborators in your writeup. Use of outside sources is strongly discouraged; if, however, you do use an outside source, you must reference it in your solution.
Prerequisites
This is a graduate-level class and will move fairly quickly. We shall assume a significant level of mathematical maturity. The formal prerequisites are fairly minimal however; they consist of:
- Multivariable Calculus (18.02)
- Linear Algebra (18.06)
- Basic Algorithms, and
- Basic Probability
Some experience with algorithms beyond the introductory level will be helpful, but it is not strictly necessary.
Calendar
| SES # | TOPICS | KEY DATES |
|---|---|---|
| Spectral Graph Theory | ||
| 1 | Linear algebra review, adjacency and Laplacian matrices associated with a graph, example Laplacians | |
| 2 | Properties of the Laplacian, positive semidefinite matricies, spectra of common graphs, connection to the continuous Laplacian | |
| 3 | Courant-Fischer and Rayleigh quotients, graph cutting, Cheerger's Inequality | |
| 4 | (Lazy) random walks, their stationary distribution and l2-convergence, normalized Laplacian, conductance, Monte Carlo methods | |
| 5 | Monte Carlo methods continued, approximate DNF counting, approximating the permanent of 0-1 matrices | |
| 6 | Diameters and eigenvalues, expander graphs | |
| 7 | Nonblocking routing networks, local and almost-linear time clustering and partitioning, Lovasz-Simonovits Theorem | |
| 8 | Local and almost-linear time clustering and partitioning (cont.), PageRank, introduction to sparsification | |
| 9 | Sparsification (combinatorial and spectral), effective resistance, matrix pseudoinverses and tail bounds | |
| 10 | Spectral sparsification (cont.), introduction to convex geometry | Problem set 1 due |
| Convex Geometry | ||
| 11 | Polar of a convex body, separating hyperplanes, norms and convex bodies, Banach-Mazur distance, Fritz John's theorem | |
| 12 | Separating hyperplanes (cont.), Banach-Mazur distance, Fritz John's theorem, Brunn-Minkowski inequality | |
| 13 | Brunn-Minkowski inequality (cont.), Brunn's theorem, isoperimetric inequality, Grunbaum's theorem | |
| 14 | Approximating the volume of a convex body | |
| 15 | Random sampling from a convex body (cont.), grid walk, introduction to concentration of measure | |
| 16 | Concentration of measure and the isoperimetric inequality, Johnson-Lindenstrauss theorem | |
| 17 | Johnson-Lindenstrauss theorem (cont.), Dvoretsky's theorem | |
| Lattices and Basis Reduction | ||
| 18 | Lattices, fundamental parallelepiped and dual of a lattice, shortest vectors, Blichfield's theorem | |
| 19 | Minkowski's theorem, shortest/closest vector problem, lattice basis reduction, Gauss' algorithm | |
| 20 | LLL algorithm for lattice basis reduction, application to integer programming | Problem set 2 due |
| Iterative Methods for Linear Algebra | ||
| 21 | Iterative methods to solve linear systems, steepest descent | |
| 22 | Convergence analysis of steepest descent and conjugate gradients | |
| 23 | Preconditioning on Laplacians, ultra-sparsifiers | |
| Multiplicative Weights | ||
| 24 | Multiplicative weights | |
| 25 | Multiplicative weights and applications to zero-sum games, linear programming, boosting, and approximation algorithms | Problem set 3 due |