Calendar

Week 1 - Course intro

Sept 27

Lecture 1

HW1 out

Intro
Course logistics
Defining time series models
Reading:
1. S&S 1.2

Week 2,3,4 - Traditional time series methods

Oct 2

Lecture 2

CQ1 out

Stationarity, autocorrelation
Foundational stationary time series models – Part 1 (AR and MA processes)
Reading:
1. S&S 1.3-1.5; S&S 3.1 (up to ARMA)

Oct 4

Lecture 3

HW1 Part 1 (prereqs) due

CQ1 due

Foundational stationary time series models – Part 2 (ARMA processes)
Forecasting
Reading:
1. S&S 3.1 (remainder); S&S 3.3-3.4

Oct 9

Lecture 4

Estimating ARMA models
Foundational non-stationary time series models (ARIMA/SARIMA)
Multivariate processes
Reading:
1. S&S 3.5-3.7; 3.9; 5.6 (high level)
2. (optional) Lutkepohl 2.1 (VAR); 11.1-11.3 (VARMA)

Oct 11

Lecture 5

HW1 Part 2 due

HW2 out

CQ2 out

State space models (SSMs)
Kalman filtering/smoothing
Dynamic latent factor models
Reading:
1. S&S 6.1-6.2
2. Murphy 29.6-29.8.3; 8.1-8.2
3. (optional) Lutkepohl 18.1-18.4 (SSMs, filtering/smoothing)
4. (optional) Bishop 13.3 (reading 13.2 first will help)

Oct 16

Lecture 6

Hidden Markov models (HMMs)
Learning and inference in HMMs
Reading:
1. S&S 6.3, 6.9
2. Murphy 29.1-29.4.2; 9.2
3. (optional) Bishop 13.2-13.3 (cont’d); 9.2-9.3 (EM background)

Oct 18

Lecture 7

CQ2 due

Learning SSMs cont’d (EM algorithm)
Switching SSMs
Reading:
1. S&S 6.10
2. Murphy 29.9

Week 5 - Deep learning-based sequence models

Oct 23

Lecture 8

project proposal due

Refresher on feedforward neural networks, backpropagation
Autoregressive and recurrent neural networks (RNNs)
Reading:
1. GBC Ch.6; Ch.10.1-10.5 (excluding 10.2.2)
2. (optional) Murphy 16.1-16.3

Oct 25

Lecture 9

HW2 due

HW3 out

CQ3 out

Backpropagation through time
Gated RNNs (LSTMs, GRUs)
Reading:
1. GBC 10.2.2; 10.7-10.12
2. (optional) Murphy 16.1-16.3
3. (optional) Amazon research WWW 2020 tutorial on RNNs and CNNs for time series
4. (optional) Deep AR
5. (optional) N-BEATS

Oct 30

Lecture 10

Refresher on convolutional neural networks (CNNs)
CNNs for sequential data
Transformers
Reading:
1. GBC Ch. 9
2. (optional) Attention Is All You Need
3. (optional) The Illustrated Transformer
4. (optional) Transformer (Google AI blog post)
5. (optional) Transformers for time series forecasting
6. (optional) More on attention for those interested

Week 6 - Advanced topics

Nov 1

Lecture 11

CQ3 due

Representation learning for time series

Nov 6

Lecture 12

Guest Lecture: State-of-the-art sequence models – Tri Dao

Week 7,8 - Continuous-time modeling

Nov 8

Lecture 13

HW3 due

HW4 out

CQ4 out

Event-based sequence models (Poisson processes, Hawkes processes)
Reading:
1. (optional) Poisson processes
2. (optional) Hawkes processes
3. (optional) Neural Hawkes processes

Nov 13

Lecture 14

Project midway due

Guest Lecture: Continuous-time modeling via neural ODEs – Patrick Kidger
Reading:
1. (optional) Patrick Kidger thesis

Nov 15

Lecture 15

CQ4 due

Guest Lecture: Neural temporal point processes – Oleksandr Shchur
Reading:
1. (optional) Oleksandr Shchur thesis

Week 9 - Thanksgiving break

Week 10,11 - Advanced topics

Nov 27

Lecture 16 Canceled

~~Special Time: 9am PT, Guest Lecture: TBA – Oriol Vinyals~~

Nov 29

Lecture 17

HW4 due

CQ5 out

Hybrid / gray-box models, structured neural sequence models

Dec 4

Lecture 18

Course wrap-up

Dec 6

Lecture 19

CQ5 due

Poster due

Poster session

Dec 11

Project report due