A stochastic approximation view of boosting. The boosting as a stochastic approximation algorithm is considered. This new interpretation provides an alternative theoretical framework for investigation. Following the results of stochastic approximation theory a stochastic approximation boosting algorithm, SABoost, is proposed. By adjusting its step sizes, SABoost will have different kinds of properties. Empirically, it is found that SABoost with a small step size will have smaller training and testing errors difference, and when the step size becomes large, it tends to overfit (i.e. bias towards training scenarios). This choice of step size can be viewed as a smooth (early) stopping rule. The performance of AdaBoost is compared and contrasted.
Keywords for this software
References in zbMATH (referenced in 6 articles , 1 standard article )
Showing results 1 to 6 of 6.
- Zhang, Chun-Xia; Wang, Guan-Wei; Zhang, Jiang-She: An empirical bias-variance analysis of DECORATE ensemble method at different training sample sizes (2012)
- Chang, Yuan-Chin Ivan; Huang, Yufen; Huang, Yu-Pai: Early stopping in (L_2)Boosting (2010)
- Rokach, Lior: Taxonomy for characterizing ensemble methods in classification tasks: a review and annotated bibliography (2009)
- Zhang, Chun-Xia; Zhang, Jiang-She; Zhang, Gai-Ying: Using boosting to prune double-bagging ensembles (2009)
- Croux, Christophe (ed.); Gallopoulos, Efstratios (ed.); van Aelst, Stefan (ed.); Zha, Hongyuan (ed.): Editorial: Machine learning and robust data mining (2007)
- Tsao, C. Andy; Chang, Yuan-Chin Ivan: A stochastic approximation view of boosting (2007)