Time & Location: MW 4:30–5:45 PM, Online Synchronized

Instructor: Dr. Chau-Wai Wong, chauwai dot wong at ncsu.edu

Discussion Forum: ECE792-41 on Piazza

Office hours: TuTh 5:30–6:30 PM, or by appointment

Objective: (1) To prepare students with statistical foundations for signal processing (SP) and machine learning (ML) research, and (2) to engage students in real-world signal processing and machine learning tasks.

Prerequisites: Undergrad signal processing and probability/statistics. Basic programming skills. Talk to the instructor if the prerequisites are not met.

Corequisite: Random Processes (for Topics IV and V).

Followup ECE courses: ECE 542 Neural Networks (applying ML to neural nets), ECE 759 Pattern Recognition and Machine Learning (more ML algorithms), ECE 751 Detection and Estimation Theory (in-depth treatment of statistics)

Workload & Grading: There will be 6 homework assignments and 2 projects (60%), one midterm exam (20%), and one final exam/project (20%). The first project will be on ML (road accident prediction, or propose your own topic), and the second one will be on SP (choose one of the two projects from the 2019 offering). Projects are recommended to be done in Matlab, Python, or R.

Textbooks:

• S. Haykin, Adaptive Filter Theory, Eds. 3, 4, or 5, Pearson.
• M. Hayes, Statistical Digital Signal Processing and Modeling, Wiley, 1996.
• T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edition, Springer, 2009. [Online]
• G. James, D. Witten, T. Hastie and R. Tibshirani, Introduction to Statistical Learning with Applications in R, Springer, 2013. [Online]
• K. P. Murphy, Machine Learning: A Probabilistic Perspective, 2012. (Unity ID login required.)

Reference Books:

• P. Stoica, R. L. Moses, Spectral Analysis of Signals, Prentice Hall, 2005. [Online]
• I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016. [Online]
• J. L. Devore, Probability and Statistics for Engineering and the Sciences, Ed. 9.
• H. Scheffe, The Analysis of Variance, Wiley, 1959.
• J. J. Faraway, Linear Models with R, Taylor & Francis, 2005. [Online]

Some scanned sections/chapters of the books listed above can be found at NC State Course Reserves.

Topics:

I. Review in linear algebra and probability theory. [2 lectures]

II. Statistics fundamentals [4 lectures]

• statistic, point estimate, method of moments, maximum likelihood (MLE), maximum a posteriori (MAP).
• linear model, least-squares (LS), orthogonality principle, normal equations, variance–bias tradeoff.

III. Foundations of machine learning [10 lectures]

• curse of dimensionality, training vs. generalization errors, interpretability vs. prediction accuracy
• regression vs. classification, generalized linear models, exponential family.
• model selection: cross-validation (CV), analytical methods (AIC, BIC, MDL).
• bootstrap, bagging, regularization.

IV. Time series modeling and prediction [6 lectures]

• autoregressive (AR), moving average (MA), ARMA models. Wold decomposition. Yule–Walker equations.
• discrete wiener filtering, lattice prediction filter.
• recurrent neural networks (RNN): long short-term memory (LSTM).

V. Spectral and Frequency Estimation [5 lectures]

• nonparametric methods: periodograms and windowing methods, minimum-variance spectral estimation.
• parametric methods: ARMA; maximum entropy method; subspace approaches: Pisarenko, MUSIC, ESPRIT.