ECE 411 Introduction to Machine Learning

Fall 2022 Course Syllabus

Time & Location: Lecture: MW 4:30–5:45 pm, 1228 EB2. Discussion: F 12:50–1:40 pm, 1226 EB2.

Instructor: Dr. Chau-Wai Wong, chauwai.wong [ ] ncsu [ ] edu

Office hour: 9-9:30 am on Thursdays and 2-2:30 pm on Sundays, or by appointment via a Piazza message. Use the course's unique Zoom link for the meeting.

Teaching Assistant: Mr. Kai Yue, kyue [ ] ncsu [ ] edu

Office hour: 12:30-1:30 pm on Mondays @ 2075 EB2

Course Description: Deep learning progressed remarkably over the past decade. This course introduces fundamental concepts and algorithms in machine learning that are vital for understanding state-of-the-art and cutting-edge development in deep learning. This course exposes students to real-world applications via well-guided homework programming problems and projects. Topics include, but are not limited to regression, classification, support vector machines, crossvalidation, and convolutional neural networks (CNN), long short-term memory (LSTM), and transformers.

Prerequisites: ST 300-level or above, and ECE301/CSC316/ISE361/MA341. Talk to the instructor if prerequisites can be waived.

Course Structure: Course Structure: The course consists of two 75-min lectures and one 50-min discussion section per week. A teaching assistant will lead the discussion section, covering practice problems and answering questions from students. There will be weekly homework assignments (30%) that contain both written problems and programming problems, two midterm exams (20%×2), and one term project (30%). Programming will be in Python, R, or Matlab. Students are expected to be able to write computer programs and have mathematical maturity in probability theory (e.g., have taken a 300-level statistics course) before taking the course. A linear algebra course such as MA305/405 is recommended while taking the course.

Course Forum: ECE411 on Piazza

Homework Submission: Gradescope (access via Moodle)

Textbooks:

G. James, D. Witten, T. Hastie and R. Tibshirani, Introduction to Statistical Learning with Applications in R, Springer, 2013. [Online]
T. Hastie, R. Tibshirani, J. Friedman, The Elements of Statistical Learning, (12th Printing, 2017), 2nd Ed., Springer. [Online]
I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016. [Online]

Reference Books:

M. P. Deisenroth, A. A. Faisal, and C. S. Ong, Mathematics for Machine Learning, 2020. [Online]
S. H. Chan, Introduction to Probability for Data Science, 2021. [Online]
J. L. Devore, Probability and Statistics for Engineering and the Sciences, Ed. 9.
A. Leon-Garcia, Probability, Statistics, and Random Processes for Electrical Engineering, Ed. 3, 2008.
C. E. McCulloch, S. R. Searle, Generalized, Linear, and Mixed Models, Wiley & Sons, 2001.
K. P. Murphy, Machine Learning: A Probabilistic Perspective, 2012. [Online via NC State library]
R. O. Duda, P. E. Hart, D. G. Stork, Pattern Classification, 2nd Ed., Wiley, 2001.
E. Alpaydin, Introduction to Machine Learning, 2nd Ed., MIT Press, 2009.

Topics: Linear statistical models, Bayesian classifiers, support vector machine (SVM), clustering, principal component analysis (PCA), naive Bayes, topic model, hidden Markov model (HMM), convolutional neural networks (CNN), long short-term memory (LSTM), and transformers.

Class #	Date	Topic	Lecture notes	Readings	HW Assignment
1	8/22	Introduction		Video: Can We Build a Brain? ISLR Ch1–2; ML Supp	HW1 (Due 8/29)
2	8/24	Machine learning overview	Slide deck 1	ISLR Ch1–2
3	8/29	Supervised learning		ISLR Ch1–2	HW2 (due 9/12)
4	8/31	Linear regression, Matrix-vector form; Least squares; Linear algebra		Scheffe Ch1 Scheffe App 1
Deep Learning
	9/5	Labor Day
5	9/7	Linear algebra (cont'd); Geometric interpretation			HW3 (due 9/21)
6	9/12	Modern ML applications - CNN	Slide deck 2	DL Ch6, Ch9
7	9/14	Modern ML applications - LSTM		DL Ch10	HW4 (due 9/28)
8	9/19	Modern ML applications - BERT		Attention by Futrzynski
9	9/21	Neural network training: Backpropagation		DL Ch8, Ch11
Linear Statistical Models: Regression
10	9/26	Backpropagation(cont'd); Regression function	HT Ch2	ISLR Ch2; Devore 3.3, 4.2, 2.4; Leon 3.2, 5.7
11	9/28	Conditional expectation, Probability theory review		ISLR Ch2; Devore CH6
12	10/3	Exam 1			Exam 1 HW5 (due 10/17)
13	10/5	Curse of dimensionality, Model accuracy, Bias-variance trade-off	HT Ch2	ISLR Ch2
	10/10	Fall Break
14	10/12	Confidence interval	HT Ch3	ISLR 3.1, Devore 7.1, 8.1, 8.3	HW6 (due 10/26)
15	10/17	Project intro; Hypothesis test	HT Ch3		Project
16	10/19	Multiple regression, F-statistic Qualitative predictors, Interaction	HT Ch3	ISLR 3.2, ESL 3.2 ISLR 3.3.1–2	HW7 (due 11/2)
Classification
17	10/24	Logistic regression	HT Ch4	ISLR 4.1–3, ESL 4.4
18	10/26	MLE, Invariance principle		Devore 6.2	HW8 (due 11/14, postponed due to wellness day)
19	10/31	Link function for GLM; Linear discriminant analysis	HT Ch4	McCulloch 5.1–4; ISLR 4.4, Leon 6.3.1, 6.4
20	11/2	LDA (cont'd), Error types, ROC, AUC, EER	HT Ch4	ISLR 4.4.3, Murphy 5.7.2.1
21	11/7	Naive Bayes; Logistic vs. LDA		ESL 6.6.3, Murphy 3.5; ISLR 4.5; Devore 6.2	HW9 (due 11/23)
Other Topics
22	11/9	Cross-Validation	HT Ch5	ISLR 5.1
23	11/14	Cross-Validation (cont'd); Bootstrap	HT Ch5	ISLR 5.1; 5.2
24	11/16	Bootstrap (cont'd); Regularization	HT Ch5; HT Ch6	ISLR 5.2; 6.2
25	11/21	Exam 2			Exam 2
	11/23	Thanksgiving Holiday
26	11/28	Regularization (cont'd)	HT Ch9	ISLR 6.2
27	11/30	Support vector machine (SVM)	HT Ch9	ISLR Ch9
28	12/5	Unsupervised learning: PCA; K-Means; HMM; Topic model	HT Ch10	ISLR 10.2, 10.3; Topic model 1, 2

ECE 411 Introduction to Machine Learning (Fall 2022). [Fall 2021] [Fall 2019]