Coursework

University of Southern California - Electrical and Computer Engineering (Machine Learning & Data Science)

• EE 503: Probability for Electrical and Computer Engineers (Spring 2022)
  Outline
1. Truth table; Sigma-algebra(CUT); Probability measure(CAT); De. Morgans; Injective, Surjective & Bijective
2. Conditional Prob.; Total Prob.; Baye's Theorem; Boole's Inequality; Multiplication Theorem; Naive Bayes
3. Cartesian Product; Gamma Function; Pascal's Formula; Bi&Multi-nomial Theorem; Prob. Continuity; Borel Cantelli Lemma
4. Archimedian Property; Bolzano·Weierstrass Theorem; Cauchy Sequence; i.i.d.; Binomial; Hypergeometric; Negative Binomial; Geometric; Poisson; Stirling's Approx
5. Random Variable(PAM); CDF; Uniform Convergence; Beta; Cauchy; Gaussian & its Integral; Signal Detection; Exponential; Bayesian Inference
6. Expectations & Moments & Variance of Random Variables; Unconscious Statistician; Sample & Population's E & V
7. Joint & Conditional PDF/CDF; Random Process; Jointly Normal; Vector population covariance; Correlation; Uncertainty Principle; Holder's Inequality
8. Convergence in ueopd(mpd); Law of Large Numbers; Sampling Statistics; Markov Inequality; Chebyshev Inequality; Consistency; Bias of r.v.; Mean-Square Error
9. Covatiance Matrix; Whitening Process; Total Expectation/Variance; Conditional Expectation/Variance; Doubly Random Sum; Maximum Likelihood Estimation; MLE Invariance Principle
10. Sampling from a Distribution; Transform of r.v.; Monte Carlo Simulation; Monte Carlo Integration; Importance Sampling; Information Theory; Entropy; Mutual Information
11. Central Limit Theorem; Berry-Essen Theorem; Continiuty Correction; Confidence Intervals; Moment Generating Function; Characteristic Function; Levy's Continuity Theorem; Inversion Theorem
12. Bond Present Value; Rational Asset Pricing(RAP); Random/Bankruptcy RAP; 2-stage RAP; Martingale; 10-k Form
13. Markov Chains; Discrete Time and Finite Space(DTFS); Irreducible & Aperiodic; Stationary PDF; Optimal Estimators
14. Simple Linear Regression; Least Squares; Multiple Regression(OLS)
• EE 510: Linear Algebra for Engineering (Spring 2022)
  Outline
1. Vectors & Scalers; Schwerz & Triangular Inequality; Hyperplanes & Lines
2. Matrices; Hermitian, Unitary & Normal; Linear Equations; Elementary Operations
3. Echelon Forms; Gauss Elimination; Row Canonical Form; Gauss-Jordan; Homogeneous & Non-homogeneous
4. LU Decomposition; Determinant; Cofactors; Laplace Expansion; Classical Adjoint; Cramer's Rule; Principle Minors
5. Volumn & Cross Product; Permutation Matrix; Left/Right Inverse; Matrix Norm; Matrix Condition Number
6. Vector Spaces; Spanning Sets; Subspace
7. Intersection of Subspaces; Row Space; Column Space; Linear Dependence; Basis; Dimension
8. Four Fundamental Subspaces; Zero Null Space; Rank plus Nullity
9. Linear/Affine Function; Invariant Subspace; Isomorphism; Inner Product Space; Characteristic Polynomial; Cayley-Hamilton
10. Eigenvalues & Eigenvectors; Algebraic/Geometric Multiplicity; Diagonalization; Minimal Polynomial
11. Monic; Jordan Block; Companion Matrix; Block Triangular Matrix
12. Diagonal & Block Diagonal; Jordan Matrix; Nilpotent; Jordan Basis;
13. Differential Equations; Stability; Difference Equations; Fourior Coefficient or Component; Gram-Schmidt Orthogonalization; Least Square Problem
14. QR Decomposition; Positive (semi-)Definite; Negative (semi-)Definite; Local Maximum/Minimum; Hessian Matrix; Singular Value Decomposition(SVD)
15. Kronecker Product; Kronecker Sum; Vandermonde Matrix; Lagrange Interpolation; Sparse Matrix; Toeplitz Matrix
• EE 559: Machine Learning I: Supervised Methods (Spring 2022)
  Outline
1. Introduction to Machine Learning: Supervised, Semi-supervised, Unsupervised, Reinforcement, Transfer
2. Basic Concepts: Preprocessing, Feature Engineering, Learning, Prediction
3. Regression & Classification; Nearest-means classifier; OvR, OvO, Maximal Value Method(MVM); Computational Complexity
4. Feature/Weight Space; Indicator Function; Criterion Function; Optimization: Gradient Descent
5. Perceptron Learning; MSE for regression: Least Squares; MSE for classification
6. Degree of Freedom; Constraints; Regularization: Ridge Regression; Lagrangian Optimization; Kernels
7. Support Vector Machine; Slack Variables; Cross Validation; Support Vector Regression
8. ANN; Proof for ANNs as universal function approximators
9. ANN; RBF networks; K-means Clustering; Principal Components Analysis(PCA)
10. Fisher’s Linear Discriminant(FLD); Multivariate Normal; Mahalanobis Distance; Orthonormal/Whitening Transformation
11. Bayes Decision Theory; Minimum-error Classifiers; Minimum-risk Criterion; Density Estimation(Non-parametric): KDE, kNN
12. kNN Classification & Regression; Parameter Estimation: MLE, MAP, Bayesian regression
• EE 541: A Computational Introduction to Deep Learning (Fall 2022)
  Outline
1. Estimation Theory; KL-Expansion; Linear/Affine MMSE; Gaussian Random Vectors; LMS Algorithm
2. Regression and maximum likelihood; Regularization
3. Decision Theory; Logistic regression; multilayer perceptrons (MLPs)
4. Universal Approximation; MLP backpropagation (scalar, vector); Activations; Optimizers; dropout, batch normalization
5. PyTorch Practice; Vanishing Gradient; Parameter Initialization; Loss Functions; Learning Rate Scheduler
6. Dealing with Data; Convolutional Neural Networks
• EE 547: Applied and Cloud Computing for Electrical Engineers (Fall 2022)
  Outline
1. Architecture (local vs. distributed), containers, virtualization, and cloud PaaS; Create a backend
2. Overview of JavaScript and Node.js
3. Backend development: Node.js and express middleware
4. Frontend overview: HTML, CSS, JavaScript
5. Databases: NoSQL, Document, MongoDB; Rational, MySQL, Maria, Postgres
6. AWS and CLI; VPC standup
7. Serverless compute; cloud storage; Application scalability
8. Frontend development and application frameworks
9. Backend API patterns: REST vs GraphQL
10. Authentication and Access control
11. Lifecycle: testing, continuous deployment, maintenance
• EE 660: Machine Learning II: Mathematical Foundations and Methods (Fall 2022)
  Outline
1. Regression and Regularization: Lasso, Ridge, Bridge, Logistic, Linear Regression
2. Tree: CART and Ensemble of trees
3. Bagging: Bootstrap Aggregating; Random Forest
4. Boosting (Exponential loss & Forward Stagewise Additive Modeling) and Adaboost
5. Learning Theory I: In/Out-sample error, Hoeffding Inequality, loose bound, Generalization Error, Dichotomy, Growth Function, Break Points, VD Dimension
6. Learning Theory II: VC Generalization Bound, Bound & Assumptions, Bias & Variance, Overfitting (Stochastic & Deterministic Noise), Regularization, Multi-class case
7. Learning Theory III: Occam's Razor; Axiom of Non-Falsifiability; Sampling Bias; Data Snooping; Regression Case Error Bound
8. Transfer Learning I: Notation, Symmetric difference hypothesis divergence (SDHD), Cross Domain Generalization Error, General version of Error, Empirical measurement of SDHD
9. Transfer Learning II: Common Data Shifts (Prior, Covariance, Concept shifts), Importance Weighting, TrAdaBoost, Subspace Alignment (SA), TL modes (Inductive, Transductive, Unsupervised), TL - Knowledge to transfer (Instances, feature representations, parameters)
10. Semi-Supervised Learning I: Assumptions, Inductive & Transductive SSL, Self-Training, Mixture Models, Expectation Maximization
11. Semi-Supervised Learning II: Semi-Supervised Support Vector Machine (S3VM), Graph-based techniques (Mincut algo, Harmonic Function algo, Manifold regularization), Co-Training
12. Unsupervised Learning: Mixture Models, K-means Clustering, K-medoids Clustering (PAM algo, Voronoi iteration), Similarity & Dissimilarity Measures, Hierarchical Clustering, Dandogram, Nearest/Farthest Neighbor Algo, Choose K (BIC, Stepwise, CH index)
13. Human Interpretability: Intrinsic & Post-hoc, Model-agnostic & model-specific, Globally & Locally Interpretable, Mimic Learning, Partial Dependence Plot, Individual Conditional Expectation (ICE), Local interpretable model-agnostic explanations (LIME)

China Agricultural University - Computer Science and Technology

This is a subset of the advanced classes I took at CAU.

Mathematics	Core Courses	Hardware	Selectives
Discrete Mathematics Ⅰ Discrete Mathematics Ⅱ Discrete Mathematics Ⅲ Computer Graphics	Data Structure Algorithms Design and Analysis Computer Organization & Architecture Principles of Database Systems Operating System Computer Networks Compiling Principle Software Engineering Artificial Intelligence	Interface Technology Digital Electronic Technology Analog Electronic Technology	Virtual Reality Technology Digital Image Processing Data Mining Android Development