Benchmarking network embedding models for link prediction : are we making progress?
- Author
- Alexandru-Cristian Mara (UGent) , Jefrey Lijffijt (UGent) and Tijl De Bie (UGent)
- Organization
- Project
- Abstract
- Network embedding methods map a network's nodes to vectors in an embedding space, in such a way that these representations are useful for estimating some notion of similarity or proximity between pairs of nodes in the network. The quality of these node representations is then showcased through results of downstream prediction tasks. Commonly used benchmark tasks such as link prediction, however, present complex evaluation pipelines and an abundance of design choices. This, together with a lack of standardized evaluation setups can obscure the real progress in the field. In this paper, we aim to shed light on the state-of-the-art of network embedding methods for link prediction and show, using a consistent evaluation pipeline, that only thin progress has been made over the last years. The newly conducted benchmark that we present here, including 17 embedding methods, also shows that many approaches are outperformed even by simple heuristics. Finally, we argue that standardized evaluation tools can repair this situation and boost future progress in this field.
- Keywords
- representation learning, graph embedding, network embedding, link prediction, evaluation
Downloads
-
DS352.pdf
- full text (Accepted manuscript)
- |
- open access
- |
- |
- 1.37 MB
-
(...).pdf
- full text (Accepted manuscript)
- |
- UGent only
- |
- |
- 565.15 KB
-
(...).pdf
- full text (Published version)
- |
- UGent only
- |
- |
- 740.40 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-8675297
- MLA
- Mara, Alexandru-Cristian, et al. “Benchmarking Network Embedding Models for Link Prediction : Are We Making Progress?” 2020 IEEE 7TH INTERNATIONAL CONFERENCE ON DATA SCIENCE AND ADVANCED ANALYTICS (DSAA 2020), edited by G. Webb et al., IEEE, 2020, pp. 138–47, doi:10.1109/DSAA49011.2020.00026.
- APA
- Mara, A.-C., Lijffijt, J., & De Bie, T. (2020). Benchmarking network embedding models for link prediction : are we making progress? In G. Webb, Z. Zhang, V. S. Tseng, G. Williams, M. Vlachos, & L. Cao (Eds.), 2020 IEEE 7TH INTERNATIONAL CONFERENCE ON DATA SCIENCE AND ADVANCED ANALYTICS (DSAA 2020) (pp. 138–147). https://doi.org/10.1109/DSAA49011.2020.00026
- Chicago author-date
- Mara, Alexandru-Cristian, Jefrey Lijffijt, and Tijl De Bie. 2020. “Benchmarking Network Embedding Models for Link Prediction : Are We Making Progress?” In 2020 IEEE 7TH INTERNATIONAL CONFERENCE ON DATA SCIENCE AND ADVANCED ANALYTICS (DSAA 2020), edited by G. Webb, Z. Zhang, V. S. Tseng, G. Williams, M. Vlachos, and L. Cao, 138–47. IEEE. https://doi.org/10.1109/DSAA49011.2020.00026.
- Chicago author-date (all authors)
- Mara, Alexandru-Cristian, Jefrey Lijffijt, and Tijl De Bie. 2020. “Benchmarking Network Embedding Models for Link Prediction : Are We Making Progress?” In 2020 IEEE 7TH INTERNATIONAL CONFERENCE ON DATA SCIENCE AND ADVANCED ANALYTICS (DSAA 2020), ed by. G. Webb, Z. Zhang, V. S. Tseng, G. Williams, M. Vlachos, and L. Cao, 138–147. IEEE. doi:10.1109/DSAA49011.2020.00026.
- Vancouver
- 1.Mara A-C, Lijffijt J, De Bie T. Benchmarking network embedding models for link prediction : are we making progress? In: Webb G, Zhang Z, Tseng VS, Williams G, Vlachos M, Cao L, editors. 2020 IEEE 7TH INTERNATIONAL CONFERENCE ON DATA SCIENCE AND ADVANCED ANALYTICS (DSAA 2020). IEEE; 2020. p. 138–47.
- IEEE
- [1]A.-C. Mara, J. Lijffijt, and T. De Bie, “Benchmarking network embedding models for link prediction : are we making progress?,” in 2020 IEEE 7TH INTERNATIONAL CONFERENCE ON DATA SCIENCE AND ADVANCED ANALYTICS (DSAA 2020), Sydney, Australia, 2020, pp. 138–147.
@inproceedings{8675297,
abstract = {{Network embedding methods map a network's nodes to vectors in an embedding space, in such a way that these representations are useful for estimating some notion of similarity or proximity between pairs of nodes in the network. The quality of these node representations is then showcased through results of downstream prediction tasks. Commonly used benchmark tasks such as link prediction, however, present complex evaluation pipelines and an abundance of design choices. This, together with a lack of standardized evaluation setups can obscure the real progress in the field. In this paper, we aim to shed light on the state-of-the-art of network embedding methods for link prediction and show, using a consistent evaluation pipeline, that only thin progress has been made over the last years. The newly conducted benchmark that we present here, including 17 embedding methods, also shows that many approaches are outperformed even by simple heuristics. Finally, we argue that standardized evaluation tools can repair this situation and boost future progress in this field.}},
author = {{Mara, Alexandru-Cristian and Lijffijt, Jefrey and De Bie, Tijl}},
booktitle = {{2020 IEEE 7TH INTERNATIONAL CONFERENCE ON DATA SCIENCE AND ADVANCED ANALYTICS (DSAA 2020)}},
editor = {{Webb, G. and Zhang, Z. and Tseng, V. S. and Williams, G. and Vlachos, M. and Cao, L.}},
isbn = {{9781728182063}},
issn = {{2472-1573}},
keywords = {{representation learning,graph embedding,network embedding,link prediction,evaluation}},
language = {{eng}},
location = {{Sydney, Australia}},
pages = {{138--147}},
publisher = {{IEEE}},
title = {{Benchmarking network embedding models for link prediction : are we making progress?}},
url = {{http://doi.org/10.1109/DSAA49011.2020.00026}},
year = {{2020}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: