Advanced search
1 file | 583.64 KB Add to list
Author
Organization
Abstract
Network embedding (NE) methods aim to learn low-dimensional representations of network nodes as vectors, typically in Euclidean space. These representations are then validated on a variety of downstream prediction tasks. Link prediction is one of the most popular choices for assessing the performance of NE methods. However, the complexity of link prediction requires a carefully designed evaluation pipeline in order to provide consistent, reproducible and comparable results. We argue this has not been considered sufficiently in many recent works. The main goal of this paper is to overcome difficulties associated with evaluation pipelines and reproducibility of results. We introduce EvalNE, an evaluation framework to transparently assess and compare the performance of NE methods on link prediction. EvalNE provides automation and abstraction for tasks such as hyper-parameter tuning, model validation, edge sampling, computation of edge embeddings and model validation. The framework integrates efficient procedures for edge and non-edge sampling and can be used to easily evaluate any off-the-shelf embedding method. The framework is freely available as a Python toolbox. Finally, demonstrating the usefulness of EvalNE in practice, we conduct an empirical study in which we try to replicate and analyse experimental sections of several influential papers.
Keywords
network embedding, link prediction, evaluation, edge sampling, negative sampling, graphs

Downloads

  • article 2.pdf
    • full text (Published version)
    • |
    • open access
    • |
    • PDF
    • |
    • 583.64 KB

Citation

Please use this url to cite or link to this publication:

MLA
Mara, Alexandru Cristian, et al. “EvalNE : A Framework for Evaluating Network Embeddings on Link Prediction.” Proceedings of the 1st Workshop on Evaluation and Experimental Design in Data Mining and Machine Learning (EDML 2019) Co-Located with SIAM International Conference on Data Mining (SDM 2019), vol. 2436, 2019, pp. 5–13.
APA
Mara, A. C., Lijffijt, J., & De Bie, T. (2019). EvalNE : a framework for Evaluating Network Embeddings on link prediction. In Proceedings of the 1st Workshop on Evaluation and Experimental Design in Data Mining and Machine Learning (EDML 2019) co-located with SIAM International Conference on Data Mining (SDM 2019) (Vol. 2436, pp. 5–13). Calgary, Canada.
Chicago author-date
Mara, Alexandru Cristian, Jefrey Lijffijt, and Tijl De Bie. 2019. “EvalNE : A Framework for Evaluating Network Embeddings on Link Prediction.” In Proceedings of the 1st Workshop on Evaluation and Experimental Design in Data Mining and Machine Learning (EDML 2019) Co-Located with SIAM International Conference on Data Mining (SDM 2019), 2436:5–13.
Chicago author-date (all authors)
Mara, Alexandru Cristian, Jefrey Lijffijt, and Tijl De Bie. 2019. “EvalNE : A Framework for Evaluating Network Embeddings on Link Prediction.” In Proceedings of the 1st Workshop on Evaluation and Experimental Design in Data Mining and Machine Learning (EDML 2019) Co-Located with SIAM International Conference on Data Mining (SDM 2019), 2436:5–13.
Vancouver
1.
Mara AC, Lijffijt J, De Bie T. EvalNE : a framework for Evaluating Network Embeddings on link prediction. In: Proceedings of the 1st Workshop on Evaluation and Experimental Design in Data Mining and Machine Learning (EDML 2019) co-located with SIAM International Conference on Data Mining (SDM 2019). 2019. p. 5–13.
IEEE
[1]
A. C. Mara, J. Lijffijt, and T. De Bie, “EvalNE : a framework for Evaluating Network Embeddings on link prediction,” in Proceedings of the 1st Workshop on Evaluation and Experimental Design in Data Mining and Machine Learning (EDML 2019) co-located with SIAM International Conference on Data Mining (SDM 2019), Calgary, Canada, 2019, vol. 2436, pp. 5–13.
@inproceedings{8627450,
  abstract     = {Network embedding (NE) methods aim to learn low-dimensional representations of network nodes as vectors, typically in Euclidean space. These representations are then validated on a variety of downstream prediction tasks. Link prediction is one of the most popular choices for assessing the performance of NE methods. However, the complexity of link prediction requires a carefully designed evaluation pipeline in order to provide consistent, reproducible and comparable results. We argue this has not been considered sufficiently in many recent works. The main goal of this paper is to overcome difficulties associated with evaluation pipelines and reproducibility of results. We introduce EvalNE, an evaluation framework to transparently assess and compare the performance of NE methods on link prediction. EvalNE provides automation and abstraction for tasks such as hyper-parameter tuning, model validation, edge sampling, computation of edge embeddings and model validation. The framework integrates efficient procedures for edge and non-edge sampling and can be used to easily evaluate any off-the-shelf embedding method. The framework is freely available as a Python toolbox. Finally, demonstrating the usefulness of EvalNE in practice, we conduct an empirical study in which we try to replicate and analyse experimental sections of several influential papers.},
  author       = {Mara, Alexandru Cristian and Lijffijt, Jefrey and De Bie, Tijl},
  booktitle    = {Proceedings of the 1st Workshop on Evaluation and Experimental Design in Data Mining and Machine Learning (EDML 2019) co-located with SIAM International Conference on Data Mining (SDM 2019)},
  issn         = {1613-0073},
  keywords     = {network embedding,link prediction,evaluation,edge sampling,negative sampling,graphs},
  language     = {eng},
  location     = {Calgary, Canada},
  pages        = {5--13},
  title        = {EvalNE : a framework for Evaluating Network Embeddings on link prediction},
  url          = {http://ceur-ws.org/Vol-2436/#article_2},
  volume       = {2436},
  year         = {2019},
}