Chapter 13 Ensemble Methods

While this chapter is currently completely incomplete, the following resources will be useful for navigating the Graduate Student quiz in the Fall 2020 semester of STAT 432. Note that these resources might not necessarily follow all conventions of STAT 432, so notation and nomenclature may have minor differences.

This chapter introduces ensemble methods that use the combination of several models fit to the same data to create one model that may perform better than any single model.

The following are old notes from STAT 432. Like the resources below, these notes suffer from some deviation of the conventions established throughout the course this semester.

13.1 Bagging

Bagging is a combination of the words bootstrap and aggregation and refers to a process of fitting many models to bootstrap resamples of data and then aggregating the predictions from these models. Most of the reason we introduced the bootstrap earlier was for its use in the creation of ensemble methods. Bagging is often associated with decision trees, but you could use any methods that you’d like!

13.2 Random Forest

A random forest is a method that combines decision trees, bagging, and a little bit of extra randomness. The randomness is added to overcome the correlation in the results of the models in the ensemble.