A genetic algorithm for interpretable model extraction from decision tree ensembles
- Author
- Gilles Vandewiele (UGent) , Kiani Lannoye, Olivier Janssens (UGent) , Femke Ongenae (UGent) , Filip De Turck (UGent) and Sofie Van Hoecke (UGent)
- Organization
- Abstract
- Models obtained by decision tree induction techniques excel in being interpretable. However, they can be prone to overfitting, which results in a low predictive performance. Ensemble techniques provide a solution to this problem, and are hence able to achieve higher accuracies. However, this comes at a cost of losing the excellent interpretability of the resulting model, making ensemble techniques impractical in applications where decision support, instead of decision making, is crucial. To bridge this gap, we present the genesim algorithm that transforms an ensemble of decision trees into a single decision tree with an enhanced predictive performance while maintaining interpretability by using a genetic algorithm. We compared genesim to prevalent decision tree induction algorithms, ensemble techniques and a similar technique, called ism, using twelve publicly available data sets. The results show that genesim achieves better predictive performance on most of these data sets compared to decision tree induction techniques & ism. The results also show that genesim's predictive performance is in the same order of magnitude as the ensemble techniques. However, the resulting model of genesim outperforms the ensemble techniques regarding interpretability as it has a very low complexity.
- Keywords
- IBCN, Decision support, Decision tree merging, Genetic algorithms, CLASSIFICATION TREES
Downloads
-
(...).pdf
- full text
- |
- UGent only
- |
- |
- 1.38 MB
-
6978 i.pdf
- full text
- |
- open access
- |
- |
- 548.50 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-8537061
- MLA
- Vandewiele, Gilles, et al. “A Genetic Algorithm for Interpretable Model Extraction from Decision Tree Ensembles.” Trends and Applications in Knowledge Discovery and Data Mining, 2017, edited by U Kang et al., vol. 10526, Springer, 2017, pp. 104–15, doi:10.1007/978-3-319-67274-8_10.
- APA
- Vandewiele, G., Lannoye, K., Janssens, O., Ongenae, F., De Turck, F., & Van Hoecke, S. (2017). A genetic algorithm for interpretable model extraction from decision tree ensembles. In U. Kang, E.-P. Lim, J. X. Yu, & Y.-S. Moon (Eds.), Trends and applications in knowledge discovery and data mining, 2017 (Vol. 10526, pp. 104–115). https://doi.org/10.1007/978-3-319-67274-8_10
- Chicago author-date
- Vandewiele, Gilles, Kiani Lannoye, Olivier Janssens, Femke Ongenae, Filip De Turck, and Sofie Van Hoecke. 2017. “A Genetic Algorithm for Interpretable Model Extraction from Decision Tree Ensembles.” In Trends and Applications in Knowledge Discovery and Data Mining, 2017, edited by U Kang, Ee-Peng Lim, Jeffrey Xu Yu, and Yang-Sae Moon, 10526:104–15. Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-67274-8_10.
- Chicago author-date (all authors)
- Vandewiele, Gilles, Kiani Lannoye, Olivier Janssens, Femke Ongenae, Filip De Turck, and Sofie Van Hoecke. 2017. “A Genetic Algorithm for Interpretable Model Extraction from Decision Tree Ensembles.” In Trends and Applications in Knowledge Discovery and Data Mining, 2017, ed by. U Kang, Ee-Peng Lim, Jeffrey Xu Yu, and Yang-Sae Moon, 10526:104–115. Cham, Switzerland: Springer. doi:10.1007/978-3-319-67274-8_10.
- Vancouver
- 1.Vandewiele G, Lannoye K, Janssens O, Ongenae F, De Turck F, Van Hoecke S. A genetic algorithm for interpretable model extraction from decision tree ensembles. In: Kang U, Lim E-P, Yu JX, Moon Y-S, editors. Trends and applications in knowledge discovery and data mining, 2017. Cham, Switzerland: Springer; 2017. p. 104–15.
- IEEE
- [1]G. Vandewiele, K. Lannoye, O. Janssens, F. Ongenae, F. De Turck, and S. Van Hoecke, “A genetic algorithm for interpretable model extraction from decision tree ensembles,” in Trends and applications in knowledge discovery and data mining, 2017, Jeju, South Korea, 2017, vol. 10526, pp. 104–115.
@inproceedings{8537061, abstract = {{Models obtained by decision tree induction techniques excel in being interpretable. However, they can be prone to overfitting, which results in a low predictive performance. Ensemble techniques provide a solution to this problem, and are hence able to achieve higher accuracies. However, this comes at a cost of losing the excellent interpretability of the resulting model, making ensemble techniques impractical in applications where decision support, instead of decision making, is crucial. To bridge this gap, we present the genesim algorithm that transforms an ensemble of decision trees into a single decision tree with an enhanced predictive performance while maintaining interpretability by using a genetic algorithm. We compared genesim to prevalent decision tree induction algorithms, ensemble techniques and a similar technique, called ism, using twelve publicly available data sets. The results show that genesim achieves better predictive performance on most of these data sets compared to decision tree induction techniques & ism. The results also show that genesim's predictive performance is in the same order of magnitude as the ensemble techniques. However, the resulting model of genesim outperforms the ensemble techniques regarding interpretability as it has a very low complexity.}}, author = {{Vandewiele, Gilles and Lannoye, Kiani and Janssens, Olivier and Ongenae, Femke and De Turck, Filip and Van Hoecke, Sofie}}, booktitle = {{Trends and applications in knowledge discovery and data mining, 2017}}, editor = {{Kang, U and Lim, Ee-Peng and Yu, Jeffrey Xu and Moon, Yang-Sae}}, isbn = {{9783319672731}}, issn = {{0302-9743}}, keywords = {{IBCN,Decision support,Decision tree merging,Genetic algorithms,CLASSIFICATION TREES}}, language = {{eng}}, location = {{Jeju, South Korea}}, pages = {{104--115}}, publisher = {{Springer}}, title = {{A genetic algorithm for interpretable model extraction from decision tree ensembles}}, url = {{http://doi.org/10.1007/978-3-319-67274-8_10}}, volume = {{10526}}, year = {{2017}}, }
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: