LAB 01
Random Variable Generation
Acceptance–Rejection
Explore how the proposal distribution, envelope coefficient, and random candidates determine which samples are accepted.
Open labSTAT3003: Fundamentals of Statistical Algorithm
Course overview
This course introduces the fundamental ideas and computational tools behind modern statistical algorithms. Topics include random-variable generation, Monte Carlo methods, optimization, Markov chain Monte Carlo, and bootstrap methods.
Course materials
Course materials will be continuously updated throughout the course.
Syllabus
Lecture 1.1: Random Variable Generation (1)
Lecture 1.2: Random Variable Generation (2)
Lecture 2.1: Monte Carlo Integration
Lecture 2.2: Monte Carlo Inference
Lecture 2.3: Monte Carlo Variance Reduction
Lecture 2.4: Markov Chain Monte Carlo
Lecture 3.1: Root Finding, Newton Methods, and Fisher Scoring
Labs
LAB 02Copula Models
Copula Model
Choose a copula and marginal distributions to explore dependence, tail behavior, and scatterplots through inverse transforms.
Open labLAB 03Monte Carlo Inference
Power of Hypothesis Testing
Compare mean tests across normal, heavy-tailed, and skewed distributions. Explore how the population mean and sample size change rejection rates and power.
Open labLAB 04Numerical Optimization
Bisection & Newton’s Method
Step through the lecture example: shrink a bracket with bisection, or follow tangent lines with Newton’s method to find the same root.
Open labLAB 05Fisher Scoring
Logistic Regression
Step through Fisher scoring to estimate beta. Follow the log-likelihood curve, local quadratic approximation, and parameter updates.
Open labCourse schedule
| Week | Date | Planned topics | Slides / module | Assignment |
|---|---|---|---|---|
| 1 | Sep 1 | Course introduction; random-number generation; inverse transform method | 0-syllabus; 1.1 | |
| 2 | Sep 8 | Inverse transform method; acceptance–rejection algorithm | 1.1 | |
| 3 | Sep 15 | Mixture models; special transformations; copula models | 1.2 | Assignment 1 |
| 4 | Sep 22 | Monte Carlo integration | 2.1 | Assignment 1 due |
| 5 | Sep 29 | Monte Carlo statistical inference: parameter estimation and hypothesis testing | 2.2 | |
| 6 | Oct 10 | Variance reduction: antithetic variables, control variates | 2.3 | Assignment 2 |
| 7 | Oct 13 | Variance reduction: importance sampling, stratified sampling | 2.3 | |
| 8 | Oct 20 | MCMC: Markov-chain foundations; Metropolis–Hastings algorithm | 2.4 | Assignment 2 due |
| 9 | Oct 27 | Gibbs sampling; Bisection method | 2.4; 3.1 | Assignment 3 |
| 10 | Nov 3 | Newton’s method; Fisher scoring | 3.1 | Assignment 3 due |
| 11 | Nov 10 | Gradient descent | 3.2 | |
| 12 | Nov 17 | Gradient descent; coordinate descent | 3.2 | Assignment 4 released |
| 13 | Nov 24 | EM algorithm | 3.3 | Assignment 4 due |
| 14 | Dec 1 | EM algorithm | 3.3 | |
| 15 | Dec 8 | MM algorithm; ADMM algorithm | 3.4 | Assignment 5 released |
| 16 | Dec 15 | ADMM algorithm | 3.4 | Assignment 5 due |
| 17 | Dec 22 | Bootstrap: bias and standard-error estimation | 4 | |
| 18 | Dec 29 | Bootstrap confidence intervals; cross-validation; course review | 4 | Assignment 6 released |
| Post-course | Jan 5, 2027 | Assignment 6 due |