Advanced search
2 files | 1.93 MB Add to list

A genetic algorithm for interpretable model extraction from decision tree ensembles

Author
Organization
Abstract
Models obtained by decision tree induction techniques excel in being interpretable. However, they can be prone to overfitting, which results in a low predictive performance. Ensemble techniques provide a solution to this problem, and are hence able to achieve higher accuracies. However, this comes at a cost of losing the excellent interpretability of the resulting model, making ensemble techniques impractical in applications where decision support, instead of decision making, is crucial. To bridge this gap, we present the genesim algorithm that transforms an ensemble of decision trees into a single decision tree with an enhanced predictive performance while maintaining interpretability by using a genetic algorithm. We compared genesim to prevalent decision tree induction algorithms, ensemble techniques and a similar technique, called ism, using twelve publicly available data sets. The results show that genesim achieves better predictive performance on most of these data sets compared to decision tree induction techniques & ism. The results also show that genesim's predictive performance is in the same order of magnitude as the ensemble techniques. However, the resulting model of genesim outperforms the ensemble techniques regarding interpretability as it has a very low complexity.
Keywords
IBCN, Decision support, Decision tree merging, Genetic algorithms, CLASSIFICATION TREES

Downloads

  • (...).pdf
    • full text
    • |
    • UGent only
    • |
    • PDF
    • |
    • 1.38 MB
  • 6978 i.pdf
    • full text
    • |
    • open access
    • |
    • PDF
    • |
    • 548.50 KB

Citation

Please use this url to cite or link to this publication:

MLA
Vandewiele, Gilles, et al. “A Genetic Algorithm for Interpretable Model Extraction from Decision Tree Ensembles.” Trends and Applications in Knowledge Discovery and Data Mining, 2017, edited by U Kang et al., vol. 10526, Springer, 2017, pp. 104–15, doi:10.1007/978-3-319-67274-8_10.
APA
Vandewiele, G., Lannoye, K., Janssens, O., Ongenae, F., De Turck, F., & Van Hoecke, S. (2017). A genetic algorithm for interpretable model extraction from decision tree ensembles. In U. Kang, E.-P. Lim, J. X. Yu, & Y.-S. Moon (Eds.), Trends and applications in knowledge discovery and data mining, 2017 (Vol. 10526, pp. 104–115). https://doi.org/10.1007/978-3-319-67274-8_10
Chicago author-date
Vandewiele, Gilles, Kiani Lannoye, Olivier Janssens, Femke Ongenae, Filip De Turck, and Sofie Van Hoecke. 2017. “A Genetic Algorithm for Interpretable Model Extraction from Decision Tree Ensembles.” In Trends and Applications in Knowledge Discovery and Data Mining, 2017, edited by U Kang, Ee-Peng Lim, Jeffrey Xu Yu, and Yang-Sae Moon, 10526:104–15. Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-67274-8_10.
Chicago author-date (all authors)
Vandewiele, Gilles, Kiani Lannoye, Olivier Janssens, Femke Ongenae, Filip De Turck, and Sofie Van Hoecke. 2017. “A Genetic Algorithm for Interpretable Model Extraction from Decision Tree Ensembles.” In Trends and Applications in Knowledge Discovery and Data Mining, 2017, ed by. U Kang, Ee-Peng Lim, Jeffrey Xu Yu, and Yang-Sae Moon, 10526:104–115. Cham, Switzerland: Springer. doi:10.1007/978-3-319-67274-8_10.
Vancouver
1.
Vandewiele G, Lannoye K, Janssens O, Ongenae F, De Turck F, Van Hoecke S. A genetic algorithm for interpretable model extraction from decision tree ensembles. In: Kang U, Lim E-P, Yu JX, Moon Y-S, editors. Trends and applications in knowledge discovery and data mining, 2017. Cham, Switzerland: Springer; 2017. p. 104–15.
IEEE
[1]
G. Vandewiele, K. Lannoye, O. Janssens, F. Ongenae, F. De Turck, and S. Van Hoecke, “A genetic algorithm for interpretable model extraction from decision tree ensembles,” in Trends and applications in knowledge discovery and data mining, 2017, Jeju, South Korea, 2017, vol. 10526, pp. 104–115.
@inproceedings{8537061,
  abstract     = {{Models obtained by decision tree induction techniques excel in being interpretable. However, they can be prone to overfitting, which results in a low predictive performance. Ensemble techniques provide a solution to this problem, and are hence able to achieve higher accuracies. However, this comes at a cost of losing the excellent interpretability of the resulting model, making ensemble techniques impractical in applications where decision support, instead of decision making, is crucial. 
To bridge this gap, we present the genesim algorithm that transforms an ensemble of decision trees into a single decision tree with an enhanced predictive performance while maintaining interpretability by using a genetic algorithm. We compared genesim to prevalent decision tree induction algorithms, ensemble techniques and a similar technique, called ism, using twelve publicly available data sets. The results show that genesim achieves better predictive performance on most of these data sets compared to decision tree induction techniques & ism. The results also show that genesim's predictive performance is in the same order of magnitude as the ensemble techniques. However, the resulting model of genesim outperforms the ensemble techniques regarding interpretability as it has a very low complexity.}},
  author       = {{Vandewiele, Gilles and Lannoye, Kiani and Janssens, Olivier and Ongenae, Femke and De Turck, Filip and Van Hoecke, Sofie}},
  booktitle    = {{Trends and applications in knowledge discovery and data mining, 2017}},
  editor       = {{Kang, U and Lim, Ee-Peng and Yu, Jeffrey Xu and Moon, Yang-Sae}},
  isbn         = {{9783319672731}},
  issn         = {{0302-9743}},
  keywords     = {{IBCN,Decision support,Decision tree merging,Genetic algorithms,CLASSIFICATION TREES}},
  language     = {{eng}},
  location     = {{Jeju, South Korea}},
  pages        = {{104--115}},
  publisher    = {{Springer}},
  title        = {{A genetic algorithm for interpretable model extraction from decision tree ensembles}},
  url          = {{http://doi.org/10.1007/978-3-319-67274-8_10}},
  volume       = {{10526}},
  year         = {{2017}},
}

Altmetric
View in Altmetric
Web of Science
Times cited: