DATA MINING Desktop Survival Guide by Graham Williams
Desktop Survival Project Home Preface List of Figures List of Tables Process . Data Graphics in R Understanding Data Preparing Data Building Models Evaluating Models Algorithms Apriori Bagging Bayes Classifier Boosting Conditional Trees Decision Trees Hierarchical Clustering Kernel Methods K-Means K-Nearest Neighbours Linear Models Logistic Regression Neural Networks Support Vector Machines Text Mining Open Products Borgelt Data Mining Suite R Rattle Weka Closed Products C4.5 Clementine Enterprise Miner Equbits Foresight GhostMiner InductionEngine ODM Statistica Data Miner TreeNet Virtual Predict Appendicies Glossary Bibliography Index

Boosting: Meta Algorithm

The Boosting meta-algorithm is a relatively efficient, simple, and easy to program learning strategy. The popular variant called AdaBoost (an abbreviation for Adaptive Boosting) has been described as the ``best off-the-shelf classifier in the world'' (attributed to Leo Breiman by Hastie et al. (2001, p. 302)). Boosting algorithms build multiple models from a dataset, using some other learning algorithm that need not be a particularly good learner. Boosting associates weights with entities in the dataset, and increases (boosts) the weights for those entities that are hard to accurately model. A sequence of models is constructed and after each model is constructed the weights are modified. The final model is then an additive model constructed from the sequence of models, each model's output weighted by some score. There is little tuning required and little is assumed about the learner used, except that it should be a weak learner! We note that boosting can fail to perform if there is insufficient data or if the weak models are overly complex. Boosting is also susceptible to noise.

Brought to you by Togaware.