ECE792-41 Statistical Methods for Signal Analytics

Time & Location: TuTh 1:30–2:45 PM @ 1226 Engineering Building II

Instructor: Dr. Chau-Wai Wong, firstname.lastname@ncsu.edu

Office hours: By appointment

Objective: (1) To introduce fundamental concepts of signal processing and statistics, and (2) to engage students in real-world signal analytics tasks.

Prerequisites: Random Processes & any DSP course. Basic programming skills. Talk to the instructor if the prerequisites are not met.

Workload & Grading: There will be 2 projects and 6 homework assignments (60%), one midterm exam (20%), and one final exam (20%). Projects are recommended to be done in Matlab, alternatively in R, Python, or C++.

Discussions: ECE792-41 on Piazza

Textbooks:

S. Haykin, Adaptive Filter Theory, Eds. 3, 4, or 5, Pearson.
M. Hayes, Statistical Digital Signal Processing and Modeling, Wiley, 1996.
P. Stoica, R. L. Moses, Spectral Analysis of Signals, Prentice Hall, 2005. [Online]
T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edition, Springer, 2009. [Online]
H. Scheffe, The Analysis of Variance, Wiley, 1959.
J. J. Faraway, Linear Models with R, Taylor & Francis, 2005. [Online]

Topics:

I. Fundamentals

Statistics: method of moments, maximum likelihood (MLE), least-squares (LS), orthogonality principle, normal equations.
Random processes: stationarity, ergodicity, power spectral density (PSD), autocorrelation function (ACF), partial autocorrelation function (PACF).
Numerical: condition number, eigendecomposition, singular-value decomposition (SVD), Levinson-Durbin algorithm, gradient descent, Newton's method, Quasi-Newton methods.
Model selection: cross-validation (CV), analytical methods (AIC, BIC, MDL, etc.)

II. Signal Modeling and Optimum Filtering

Time series models: autoregressive (AR), moving average (MA), ARMA. Yule-Walker equations, Wold decomposition.
Discrete Wiener filtering: forward and backward linear predictions.
Lattice prediction filter, joint-process estimation.

III. Adaptive Filtering

Least-mean-squares (LMS) algorithm.
Recursive least-squares (RLS) algorithm.

IV. Spectral and Frequency Estimation

Nonparametric methods: periodograms and windowing methods, minimum-variance spectral estimation (Capon), amplitude and phase estimator (APES)*, iterative adaptive approach (IAA)*.
Parametric methods: AR, MA, and ARMA spectral estimation; maximum entropy method.
High-resolution subspace approaches: Pisarenko, MUSIC, ESPRIT.

V. Analysis of Variance (ANOVA)

Estimable functions, Gauss-Markoff theorem.
Confidence ellipsoids/intervals, t-test, F-test.
ANOVA.

* Topics not covered due to time constraints.

	Date	Topic	Lecture notes	Assignment
Class 1	1/9	Review on probability	Handwritten
Class 2	1/11	Review on statistics	Handwritten	HW1
Class 3	1/16	Review on random processes, Discrete-time stochastic processes	Handwritten, Part1 Sec1
	1/18	Class canceled due to snow
Wiener Filtering
Class 4	1/23	Discrete-time stochastic processes	Part1 Sec1
Class 5	1/25	Autoregressive–moving-average (ARMA) model	Part1 Sec1 (cont'd)
Class 6	1/30	ARMA model (cont'd)	Part1 Sec1 (cont'd)	HW2
Class 7	2/1	Normal equations and geometric interpretation	Handwritten
Class 8	2/6	Discrete Wiener filtering	Part1 Sec2
Class 9	2/8	Linear prediction	Part1 Sec3	HW3
Class 10	2/13	Levinson-Durbin recursion	Part1 Sec4
Class 11	2/15	Lattice predictor	Part1 Sec5
Spectrum and Frequency Estimation
Class 12	2/20	Spectrum estimation	Part3 Sec1	Project 1, Dataset (due 3/15)
Class 13	2/22	Periodogram and its variants, Minimum variance (Capon)	Part3 Sec1
Class 14	2/27	Maximum entropy (MESE), Durbin's method	Part3 Sec2
Class 15	3/1	Midterm exam
		Spring Break
Class 16	3/13	Interim presentations, Midterm follow ups
Class 17	3/15	Model selection: Overview & Cross-valiation (Hastie et al. Ch7)	Handwritten
Class 18	3/20	Model selection: Analytical methods (Hastie et al. Ch7)	Handwritten	HW4
Class 19	3/22	Subspace frequency estimation methods	Part3 Sec3
Adaptive Filtering
Class 20	3/27	Adaptive signal processing introduction	Handwritten
Class 21	3/29	Least-mean-squares (LMS) algorithm, LMS convergence	Handwritten
Class 22	4/3	LMS convergence (cont'd), Matrix calculus basics, Optimization intro	Handwritten	HW5
Class 23	4/5	Gradient descent, Newton's method	Handwritten	Project 2, Dataset (due 5/1)
Class 24	4/7	Quasi-Newton methods	Handwritten
Class 25	4/7	Recursive least-squares (RLS), RLS convergence	Handwritten
Analysis of Variance (ANOVA)
Class 26	4/10	Linear statistical models intro, Estimable functions	Handwritten	HW6
Class 27	4/12	Gauss-Markoff theorem, Side conditions on parameters	Handwritten
Class 28	4/24	Confidence ellipsoids/intervals, t-test, F-test	Handwritten
Class 29	4/26	Analysis of Variance (ANOVA)	Handwritten