Lecture supplements

• Introduction to Classification and Regression
• Bayes Classifiers
• R commands for example introduced on p. 17 of HTF

• Ordinary Least Squares Regression: a Look at the Basics Without Matrices
• R commands for cherry tree regression example
• R commands for chemical engineering regression example (I didn't go over this in class, but I want you to go through it on your own and ask questions about it in class, if needed. It demonstrates some of the things that I lectured about, but didn't include in the cherry tree example that I went over.)

• LDA (Note: The pages are in the correct order, although the page numbers are incorrect.)
• QDA and RDA
• Logistic Regression
• R commands for example showing how to use LDA, QDA, RDA, and logistic regression for classification
• R commands for example showing how to use stepwise LDA for classification

• Introduction to Local Methods for Regression and Classification
• R commands for example introduced on p. 17 of HTF (same file used for Lecture 1, only I've modified it since then)
• Regression Tactics: Shrinkage Methods and Derived Input Directions
• R commands for example showing how to use the lasso, ridge regression, and principal components regression
• Comparison of Linear Methods for Classification
• R commands for example showing how to use LDA, QDA, RDA, and logistic regression for classification (same file used for Lecture 3)

• Understanding CART (Note: There is a mistake in the corner of p. 23 --- it should be 3*99 = 297 instead of (99)^3.)

• Basis Functions
• Linear Splines
• Cubic Splines

• Smoothing Splines
• MARS Basics
• R commands for example showing how to use linear splines, cubic splines, natural splines, and smoothing splines for regression with a single explanatory variable (Here is the data file used in the example.)
• R commands for example showing how to use natural cubic splines in a multiple regression setting (Here is some information about the data used in the example.)

• Density Estimation Methods for Classification (I'll say a bit more about the mixture model method in class, but still not give too many of the details.)
• R commands for example introduced on p. 17 of HTF (same file used for Lecture 1 and Lecture 4, only I've modified it since then)

• MSPE & Irreducible Error
• Mallows' C_p
• Kernel Methods for Regression
• R commands for C_p study & local regression methods
• more R commands for C_p study & local regression methods (similar to the demo above, but data is slightly different)
• R commands for example introduced on p. 17 of HTF (same file used for Lecture 1 and Lecture 4, only I've modified it since then to include more classification methods (covered in 6/30 and 7/7 classes))

• Bagging
• Random Forests

• Separating Hyperplanes to Support Vector Machines
• R commands for example introduced on p. 17 of HTF (same file used for Lecture 1, Lecture 4, and Lecture 9, only I've modified it since then to include classification using support vector machines)
• Projection Pursuit Regression
• R commands for PPR, C_p study & local regression methods (I modifed the demo found under Lecture 9 above to include projection prusuit regression.)
• Neural Network Models (Note: There is a mistake on the 2nd line from the bottom on p. 6 --- it should be just v_i instead of v_i - α.)

• Boosting
• R commands for a simple example of using the adaboost function
• R commands for example introduced on p. 17 of HTF (same file used for Lecture 1, Lecture 4, Lecture 9, and Lecture 11, only I've modified it since then to include boosting (although it may not be working correctly))
• three related presentations pertaining to TreeNet (which are best read in the order listed below)
  1. Forward Stagewise Additive Modeling
  2. MART
  3. Classification with TreeNet