Predicting inference time and energy consumption in deep learning using MLPs
- Author
- Chengjin Lyu (UGent) , Mohsen Nourazar (UGent) and Bart Goossens (UGent)
- Organization
- Project
- Abstract
- With the rapid evolution of deep learning technologies, the efficient deployment of models in real-time and energy sensitive environments has become increasingly vital. Accurately predicting how deep models consume resources ensures that these models operate within the constraints of their intended scenarios. This is particularly essential for applications such as Neural Architecture Search (NAS), edge computing, and distributed systems. Most current work focuses on predicting layer-wise inference time and energy consumption, but the summed predictions often do not align with the actual runtime and energy cost. This paper introduces a framework using two-stage Multi-Layer Perceptrons (MLPs) to predict model-wise inference time and energy consumption of deep neural networks. The first stage of our approach involves layer-wise inference time and energy usage predictions tailored to various types of neural network layers. Building upon these initial outputs, the second-stage predictor employs another MLP to aggregate these layer-wise estimations into a comprehensive prediction of the overall model’s performance. We validate the effectiveness of the proposed method on two real computing platforms. This framework can enhance the design and deployment of deep learning architectures, by accurately estimating model-wide inference time and energy consumption.
Downloads
-
ISPA2024 LYU camera ready.pdf
- full text (Accepted manuscript)
- |
- open access
- |
- |
- 131.25 KB
-
(...).pdf
- full text (Published version)
- |
- UGent only
- |
- |
- 191.03 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01J8FKR1X5HV77NA0D975TBR9N
- MLA
- Lyu, Chengjin, et al. “Predicting Inference Time and Energy Consumption in Deep Learning Using MLPs.” 22nd IEEE International Symposium on Parallel and Distributed Processing with Applications (IEEE ISPA 2024), Proceedings, IEEE, 2024, pp. 990–96, doi:10.1109/ISPA63168.2024.00131.
- APA
- Lyu, C., Nourazar, M., & Goossens, B. (2024). Predicting inference time and energy consumption in deep learning using MLPs. 22nd IEEE International Symposium on Parallel and Distributed Processing with Applications (IEEE ISPA 2024), Proceedings, 990–996. https://doi.org/10.1109/ISPA63168.2024.00131
- Chicago author-date
- Lyu, Chengjin, Mohsen Nourazar, and Bart Goossens. 2024. “Predicting Inference Time and Energy Consumption in Deep Learning Using MLPs.” In 22nd IEEE International Symposium on Parallel and Distributed Processing with Applications (IEEE ISPA 2024), Proceedings, 990–96. IEEE. https://doi.org/10.1109/ISPA63168.2024.00131.
- Chicago author-date (all authors)
- Lyu, Chengjin, Mohsen Nourazar, and Bart Goossens. 2024. “Predicting Inference Time and Energy Consumption in Deep Learning Using MLPs.” In 22nd IEEE International Symposium on Parallel and Distributed Processing with Applications (IEEE ISPA 2024), Proceedings, 990–996. IEEE. doi:10.1109/ISPA63168.2024.00131.
- Vancouver
- 1.Lyu C, Nourazar M, Goossens B. Predicting inference time and energy consumption in deep learning using MLPs. In: 22nd IEEE International Symposium on Parallel and Distributed Processing with Applications (IEEE ISPA 2024), Proceedings. IEEE; 2024. p. 990–6.
- IEEE
- [1]C. Lyu, M. Nourazar, and B. Goossens, “Predicting inference time and energy consumption in deep learning using MLPs,” in 22nd IEEE International Symposium on Parallel and Distributed Processing with Applications (IEEE ISPA 2024), Proceedings, Kaifeng, China, 2024, pp. 990–996.
@inproceedings{01J8FKR1X5HV77NA0D975TBR9N,
abstract = {{With the rapid evolution of deep learning technologies, the efficient deployment of models in real-time and energy
sensitive environments has become increasingly vital. Accurately predicting how deep models consume resources ensures that these models operate within the constraints of their intended scenarios. This is particularly essential for applications such as Neural Architecture Search (NAS), edge computing, and distributed systems. Most current work focuses on predicting layer-wise inference time and energy consumption, but the summed predictions often do not align with the actual runtime and energy cost.
This paper introduces a framework using two-stage Multi-Layer Perceptrons (MLPs) to predict model-wise inference time and energy consumption of deep neural networks. The first stage of our approach involves layer-wise inference time and energy usage predictions tailored to various types of neural network layers. Building upon these initial outputs, the second-stage predictor employs another MLP to aggregate these layer-wise estimations into
a comprehensive prediction of the overall model’s performance. We validate the effectiveness of the proposed method on two real computing platforms. This framework can enhance the design and deployment of deep learning architectures, by accurately estimating model-wide inference time and energy consumption.}},
author = {{Lyu, Chengjin and Nourazar, Mohsen and Goossens, Bart}},
booktitle = {{22nd IEEE International Symposium on Parallel and Distributed Processing with Applications (IEEE ISPA 2024), Proceedings}},
isbn = {{9798331509712}},
issn = {{2158-9208}},
language = {{eng}},
location = {{Kaifeng, China}},
pages = {{990--996}},
publisher = {{IEEE}},
title = {{Predicting inference time and energy consumption in deep learning using MLPs}},
url = {{http://doi.org/10.1109/ISPA63168.2024.00131}},
year = {{2024}},
}
- Altmetric
- View in Altmetric