ECE792-41 Statistical Methods for Signal Analytics

Time & Location: TuTh 4:30–5:45 PM @ 1229 Engineering Building II

Instructor: Dr. Chau-Wai Wong, chauwai dot wong at ncsu.edu

Office hours: 1:15–2:45 PM on Wednesdays, or by appointment

Objective: (1) To introduce fundamental concepts of signal processing and statistics, and (2) to engage students in real-world signal analytics tasks.

Course Description:

In fields such as telecommunication, bioengineering, and economics, data may naturally arise in the form of time series. To make better sense of the time-series data and exploit them for prediction, correlation along the time dimension must be explicitly analyzed and modeled. This course introduces concepts and tools of signal processing, machine learning, and statistics for time-series analytics with an emphasis on the application of the estimation theory. Students will be engaged in real-world time-series analytics problems such as physiological signals extraction and spectrum/frequency tracking.

Prerequisites: Random Processes & any DSP course. Basic programming skills. Talk to the instructor if the prerequisites are not met.

Workload & Grading: There will be 2 projects and 5 homework assignments (60%), one midterm exam (20%), and one final exam (20%). Projects are recommended to be done in Matlab, alternatively in R, Python, or C++.

Discussions: ECE792-41 on Piazza

Textbooks:

S. Haykin, Adaptive Filter Theory, Eds. 3, 4, or 5, Pearson.
M. Hayes, Statistical Digital Signal Processing and Modeling, Wiley, 1996.
P. Stoica, R. L. Moses, Spectral Analysis of Signals, Prentice Hall, 2005. [Online]
T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edition, Springer, 2009. [Online]
H. Scheffe, The Analysis of Variance, Wiley, 1959.
J. J. Faraway, Linear Models with R, Taylor & Francis, 2005. [Online]
J. L. Devore, Probability and Statistics for Engineering and the Sciences, Ed. 9.

Topics:

I. Fundamentals

Statistics: method of moments, maximum likelihood (MLE), least-squares (LS), orthogonality principle, normal equations.
Random processes: stationarity, ergodicity, power spectral density (PSD), autocorrelation function (ACF), partial autocorrelation function (PACF).
Numerical: condition number, eigendecomposition, singular-value decomposition (SVD), Levinson-Durbin algorithm, gradient descent, Newton's method, Quasi-Newton methods.
Model selection: cross-validation (CV), analytical methods (AIC, BIC, MDL, etc.)

II. Signal Modeling and Optimum Filtering

Time series models: autoregressive (AR), moving average (MA), ARMA. Yule-Walker equations, Wold decomposition.
Discrete Wiener filtering: forward and backward linear predictions.
Lattice prediction filter, joint-process estimation.

III. Adaptive Filtering

Least-mean-squares (LMS) algorithm.
Recursive least-squares (RLS) algorithm.

IV. Spectral and Frequency Estimation

Nonparametric methods: periodograms and windowing methods, minimum-variance spectral estimation (Capon), amplitude and phase estimator (APES)*, iterative adaptive approach (IAA)*.
Parametric methods: AR, MA, and ARMA spectral estimation; maximum entropy method.
High-resolution subspace approaches: Pisarenko, MUSIC, ESPRIT.

V. Analysis of Variance (ANOVA)*

Estimable functions, Gauss-Markoff theorem.
Confidence ellipsoids/intervals, t-test, F-test.
ANOVA.

* Topics not covered due to time constraints.

Class #	Date	Topic	Lecture notes	Reading Assignment	HW Assignment
Class 1	1/8	Intro, Review on probability	Handwritten
Class 2	1/10	Statistics fundamentals	Handwritten	Devore: 5.3, 5.4, 5.5, 6.1	HW1 (due 1/22)
Class 3	1/15	Statistics fundamentals (cont'd)	Handwritten	Devore: 6.2
Class 4	1/17	Statistics fundamentals (cont'd)	Handwritten	Devore: 6.1
Class 5	1/22	Normal equations and geometric interpretation	Handwritten	Scheffe CH1: 1.1-1.3	HW2 (due 2/12)
Wiener Filtering
Class 6	1/24	Discrete-time stochastic processes	Part1 Sec1	Haykin 4ed: 1.1-1.3, 1.12, 1.14
Class 7	1/29	Discrete-time stochastic processes (cont'd)
Class 8	1/31	Autoregressive–moving-average (ARMA) model	Part1 Sec1a	Haykin 4ed: 1.5
Class 9	2/5	Autoregressive–moving-average (ARMA) model (cont'd), Recitation
Class 10	2/7	Discrete Wiener filtering	Part1 Sec2	Haykin 4ed: CH2
Class 11	2/12	Linear prediction	Part1 Sec3	Haykin 4ed: 3.1-3.3
Class 12	2/14	Levinson-Durbin recursion	Part1 Sec4	Haykin 4ed: 3.3	HW3 (due 2/28)
Class 13	2/19	Discussions for HW, Lattice predictor	Part1 Sec5	Haykin 4ed: 3.8
Class 14	2/21	Lattice predictor (cont'd)			Project 1, Dataset (due 3/19)
Spectrum and Frequency Estimation
Class 15	2/26	Spectrum estimation	Part3 Sec1	Hayes 8.1-8.3
Class 16	2/28	Periodogram and its variants, Minimum variance (Capon)
Class 17	3/5	Maximum entropy (MESE), Durbin's method	Part3 Sec2	Hayes 8.5, 4.7, 8.4
Class 18	3/7	Inclass Midterm Exam
		Spring Break
Class 19	3/19	Interim presentations, Model selection	Model Selection	Hastie: CH7
Class 20	3/21	Model selection (cont'd)			HW4 (due 4/4)
Class 21	3/26	Subspace frequency estimation methods	Part3 Sec3	Hayes 8.6	Project 2, Dataset (due 4/23)
Adaptive Filtering
Class 22	4/2	Adaptive signal processing introduction	Handwritten	Haykin 4ed: 0.2-0.5
Class 23	4/4	Method of steepest descent	Handwritten	Haykin 4ed: CH4
Class 24	4/9	Discussions for HW, Least-mean-squares (LMS) algorithm	Handwritten	Haykin 4ed: 5.2
Class 25	4/11	LMS convergence	Handwritten	Haykin 4ed: 5.4	HW5 (due 4/30)
Class 26	4/16	Recursive least-squares (RLS)	Handwritten	Haykin 4ed: 9.3
Class 27	4/16	RLS convergence	Handwritten	Haykin 4ed: 9.7
Numerical Optimization
Class 28	4/18	Matrix calculus basics, Optimization intro	Handwritten
Class 29	4/23	Gradient descent, Newton's method	Handwritten
Class 30	4/25	Quasi-Newton methods	Handwritten