ENDER - A statistical framework for boosting decision rules. Induction of decision rules plays an important role in machine learning. The main advantage of decision rules is their simplicity and human-interpretable form. Moreover, they are capable of modeling complex interactions between attributes. In this paper, we thoroughly analyze a learning algorithm, called ENDER, which constructs an ensemble of decision rules. This algorithm is tailored for regression and binary classification problems. It uses the boosting approach for learning, which can be treated as generalization of sequential covering. Each new rule is fitted by focusing on examples which were the hardest to classify correctly by the rules already present in the ensemble. We consider different loss functions and minimization techniques often encountered in the boosting framework. The minimization techniques are used to derive impurity measures which control construction of single decision rules. Properties of four different impurity measures are analyzed with respect to the trade-off between misclassification (discrimination) and coverage (completeness) of the rule. Moreover, we consider regularization consisting of shrinking and sampling. Finally, we compare the ENDER algorithm with other well-known decision rule learners such as SLIPPER, LRI and RuleFit.
Keywords for this software
References in zbMATH (referenced in 9 articles )
Showing results 1 to 9 of 9.
- Marjolein Fokkema: pre: An R Package for Fitting Prediction Rule Ensembles (2017) arXiv
- Caserta, Marco; Reiners, Torsten: A pool-based pattern generation algorithm for logical analysis of data with automatic fine-tuning (2016)
- Jawanpuria, Pratik; Nath, Jagarlapudi Saketha; Ramakrishnan, Ganesh: Generalized hierarchical kernel learning (2015)
- Amin, Talha; Chikalov, Igor; Moshkov, Mikhail; Zielosko, Beata: Classifiers based on optimal decision rules (2013)
- Amin, Talha; Chikalov, Igor; Moshkov, Mikhail; Zielosko, Beata: Dynamic programming approach to optimization of approximate decision rules (2013)
- Sikora, Marek; Gudyś, Adam: CHIRA - convex hull based iterative algorithm of rules aggregation (2013)
- Amin, Talha; Chikalov, Igor; Moshkov, Mikhail; Zielosko, Beata: Dynamic programming approach for partial decision rule optimization (2012)
- Sikora, Marek; Sikora, Beata: Improving prediction models applied in systems monitoring natural hazards and machinery (2012)
- Dembczyński, Krzysztof; Kotłowski, Wojciech; Słowiński, Roman: ENDER: a statistical framework for boosting decision rules (2010) ioport