Today

• Neural network: practice, Keras Tuner workflow.

FAQs

• Ensemble methods: bagging and boosting. Not tied to tree methods. Can use any (weak) learner: the estimator argument in the sklearn.ensemble.BaggingClassifier and sklearn.ensemble.AdaBoostClassifier functions.

• For the random forest decorrelation trick, is the random subset of predictors fixed for a tree or changing at each split?

• AdaBoost vs Gradient Boosting vs XGBoost. Gradient boosting is what we discussed in class link. AdaBoost is doing exponential weighting of each observation at each boosting iteration (see, e.g., ESL Algorithm 10.1, p339).

• HW5 (SVM) bonus question.

• Why does the gradient descent (GD) algorithm move towards a local (minimum)? Variants of GD: Keras optimizers. Today’s lecture.

• Loss function for GAN.

Announcement

• HW6 is due Mar 24 @ 11:59pm. Start early; use Slack and office hours for questions.

• UCLA Synthetic Data Symposium: https://ucla-synthetic-data.github.io/. Apr 13-14, 2023.

• UCLA OARC Deep Learning Workshop Series (using PyTorch): https://github.com/huqy/deep_learning_workshops.