Time & Location: MW 4:30–5:45 PM, Online Asynchronized

Instructor: Dr. Chau-Wai Wong, chauwai dot wong at ncsu.edu

Objective: (1) To prepare students with statistical foundations for machine learning (ML) research, and (2) to engage students in real-world machine learning tasks.

Prerequisites: Undergrad signal processing and probability/statistics. Basic programming skills. Talk to the instructor if the prerequisites are not met.

Followup ECE courses: ECE 542 Neural Networks (applying ML to neural nets), ECE 759 Pattern Recognition and Machine Learning (more ML algorithms), ECE 751 Detection and Estimation Theory (in-depth treatment of statistics)

Workload & Grading: There will be 4 homework assignments (30%), 1 term project (50%), and one midterm exam (20%).

Textbooks:

• T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edition, Springer, 2009. [Online]
• G. James, D. Witten, T. Hastie and R. Tibshirani, Introduction to Statistical Learning with Applications in R, Springer, 2013. [Online]
• K. P. Murphy, Machine Learning: A Probabilistic Perspective, 2012. (Unity ID login required.)

Reference Books:

• I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016. [Online]
• J. L. Devore, Probability and Statistics for Engineering and the Sciences, Ed. 9.
• H. Scheffe, The Analysis of Variance, Wiley, 1959.
• J. J. Faraway, Linear Models with R, Taylor & Francis, 2005. [Online]

Some scanned sections/chapters of the books listed above can be found at NC State Course Reserves.

Topics:

I. Review in linear algebra and probability theory.

II. Statistics fundamentals

• statistic, point estimate, method of moments, maximum likelihood (MLE), maximum a posteriori (MAP).
• linear model, least-squares (LS), orthogonality principle, normal equations, variance–bias tradeoff.

III. Foundations of machine learning

• curse of dimensionality, training vs. generalization errors, interpretability vs. prediction accuracy
• regression vs. classification, generalized linear models, exponential family.
• model selection: cross-validation (CV), analytical methods (AIC, BIC, MDL).
• bootstrap, bagging, regularization.