Comparing a polling and push-based approach for live open data interfaces
- Author
- Brecht Van de Vyvere (UGent) , Pieter Colpaert (UGent) and Ruben Verborgh (UGent)
- Organization
- Abstract
- There are two mechanisms for publishing live changing resources on the Web: a client can pull the latest state of a resource or the server pushes updates to the client. In the state of the art, it is clear that pushing delivers a lower latency compared to pulling, however, this has not been tested for an Open Data usage scenario where 15 k clients are not an exception. Also, there are no general guidelines when to use a polling or push-based approach for publishing live changing resources on the Web. We performed (i) a field report of live Open datasets on the European and U.S. Open Data portal and (ii) a benchmark between HTTP polling and Server-Sent Events (SSE) under a load of 25 k clients. In this article, we compare the scalability and latency of updates on the client between polling and pushing. For the scenario where users want to receive an update as fast as possible, we found that SSE excels above polling in three aspects: lower CPU usage on the server, lower latency on the client and more than double the number of clients that can be served. However, considering that users can perceive a certain maximum latency on the client (MAL) of an update acceptable, we describe in this article at which MAL point a polling interface can be able to serve a higher number of clients than pushing. Open Data publishers can use these insights to determine which mechanism is the most cost-effective for the usage scenario they foresee of their live updating resources on the Web.
- Keywords
- Web API engineering, Performance and scalability, Open Data, CONTINUOUS QUERY LANGUAGE, INTEROPERABILITY
Downloads
-
DS332 i.pdf
- full text (Accepted manuscript)
- |
- open access
- |
- |
- 550.20 KB
-
(...).pdf
- full text (Published version)
- |
- UGent only
- |
- |
- 1.20 MB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-8664304
- MLA
- Van de Vyvere, Brecht, et al. “Comparing a Polling and Push-Based Approach for Live Open Data Interfaces.” WEB ENGINEERING, ICWE 2020, edited by M. Bielikova et al., vol. 12128, Springer, 2020, pp. 87–101, doi:10.1007/978-3-030-50578-3_7.
- APA
- Van de Vyvere, B., Colpaert, P., & Verborgh, R. (2020). Comparing a polling and push-based approach for live open data interfaces. In M. Bielikova, T. Mikkonen, & C. Pautasso (Eds.), WEB ENGINEERING, ICWE 2020 (Vol. 12128, pp. 87–101). https://doi.org/10.1007/978-3-030-50578-3_7
- Chicago author-date
- Van de Vyvere, Brecht, Pieter Colpaert, and Ruben Verborgh. 2020. “Comparing a Polling and Push-Based Approach for Live Open Data Interfaces.” In WEB ENGINEERING, ICWE 2020, edited by M. Bielikova, T. Mikkonen, and C. Pautasso, 12128:87–101. Springer. https://doi.org/10.1007/978-3-030-50578-3_7.
- Chicago author-date (all authors)
- Van de Vyvere, Brecht, Pieter Colpaert, and Ruben Verborgh. 2020. “Comparing a Polling and Push-Based Approach for Live Open Data Interfaces.” In WEB ENGINEERING, ICWE 2020, ed by. M. Bielikova, T. Mikkonen, and C. Pautasso, 12128:87–101. Springer. doi:10.1007/978-3-030-50578-3_7.
- Vancouver
- 1.Van de Vyvere B, Colpaert P, Verborgh R. Comparing a polling and push-based approach for live open data interfaces. In: Bielikova M, Mikkonen T, Pautasso C, editors. WEB ENGINEERING, ICWE 2020. Springer; 2020. p. 87–101.
- IEEE
- [1]B. Van de Vyvere, P. Colpaert, and R. Verborgh, “Comparing a polling and push-based approach for live open data interfaces,” in WEB ENGINEERING, ICWE 2020, Helsinki, Finland, 2020, vol. 12128, pp. 87–101.
@inproceedings{8664304,
abstract = {{There are two mechanisms for publishing live changing resources on the Web: a client can pull the latest state of a resource or the server pushes updates to the client. In the state of the art, it is clear that pushing delivers a lower latency compared to pulling, however, this has not been tested for an Open Data usage scenario where 15 k clients are not an exception. Also, there are no general guidelines when to use a polling or push-based approach for publishing live changing resources on the Web. We performed (i) a field report of live Open datasets on the European and U.S. Open Data portal and (ii) a benchmark between HTTP polling and Server-Sent Events (SSE) under a load of 25 k clients. In this article, we compare the scalability and latency of updates on the client between polling and pushing. For the scenario where users want to receive an update as fast as possible, we found that SSE excels above polling in three aspects: lower CPU usage on the server, lower latency on the client and more than double the number of clients that can be served. However, considering that users can perceive a certain maximum latency on the client (MAL) of an update acceptable, we describe in this article at which MAL point a polling interface can be able to serve a higher number of clients than pushing. Open Data publishers can use these insights to determine which mechanism is the most cost-effective for the usage scenario they foresee of their live updating resources on the Web.}},
author = {{Van de Vyvere, Brecht and Colpaert, Pieter and Verborgh, Ruben}},
booktitle = {{WEB ENGINEERING, ICWE 2020}},
editor = {{Bielikova, M. and Mikkonen, T. and Pautasso, C.}},
isbn = {{9783030505776}},
issn = {{0302-9743}},
keywords = {{Web API engineering,Performance and scalability,Open Data,CONTINUOUS QUERY LANGUAGE,INTEROPERABILITY}},
language = {{eng}},
location = {{Helsinki, Finland}},
pages = {{87--101}},
publisher = {{Springer}},
title = {{Comparing a polling and push-based approach for live open data interfaces}},
url = {{http://doi.org/10.1007/978-3-030-50578-3_7}},
volume = {{12128}},
year = {{2020}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: