CARB-Net : Camera-Assisted Radar-Based Network for vulnerable road user detection
- Author
- Wei-Yu Lee (UGent) , Martin Dimitrievski (UGent) , David Van Hamme (UGent) , Jan Aelterman (UGent) , Ljubomir Jovanov (UGent) and Wilfried Philips (UGent)
- Organization
- Abstract
- Ensuring a reliable perception of vulnerable road users is crucial for safe autonomous driving. Radar stands out as an appealing sensor choice due to its resilience in adverse weather, cost-effectiveness, depth sensing capabilities, and established role in adaptive cruise control. Nevertheless, radar's limited angular resolution poses challenges in object recognition, especially in distinguishing targets in close proximity. To tackle this limitation, we present the Camera-Assisted RadarBased Network (CARB-Net), a novel and efficient framework that merges the angular accuracy of a camera with the robustness and depth sensing capabilities of radar. We integrate camera detection information through a ground plane feed-forward array, entangling it with the early stages of a radar-based detection network. Furthermore, we introduce a unique context learning approach to ensure graceful degradation in situations of poor radar Doppler information or unfavorable camera viewing conditions. Experimental validations on public and our proposed datasets, along with benchmark comparisons, showcase CARBNet's superiority, boasting up to a 12% improvement in mAP performance. A series of ablation studies further emphasize the efficacy of the CARB-Net architecture. Our proposed dataset is released at https://github.com/weiyulee/RadVRU.
- Keywords
- Micro-Doppler Signature, Vulnerable Road User Detection, AUTOMOTIVE RADAR
Downloads
-
(...).pdf
- full text (Accepted manuscript)
- |
- UGent only
- |
- |
- 895.71 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01JAMKQGC0SKM8RJ0VQAEPDBK0
- MLA
- Lee, Wei-Yu, et al. “CARB-Net : Camera-Assisted Radar-Based Network for Vulnerable Road User Detection.” COMPUTER VISION - ECCV 2024, PT LXIV, vol. 15122, Springer, 2024, pp. 294–310, doi:10.1007/978-3-031-73039-9_17.
- APA
- Lee, W.-Y., Dimitrievski, M., Van Hamme, D., Aelterman, J., Jovanov, L., & Philips, W. (2024). CARB-Net : Camera-Assisted Radar-Based Network for vulnerable road user detection. COMPUTER VISION - ECCV 2024, PT LXIV, 15122, 294–310. https://doi.org/10.1007/978-3-031-73039-9_17
- Chicago author-date
- Lee, Wei-Yu, Martin Dimitrievski, David Van Hamme, Jan Aelterman, Ljubomir Jovanov, and Wilfried Philips. 2024. “CARB-Net : Camera-Assisted Radar-Based Network for Vulnerable Road User Detection.” In COMPUTER VISION - ECCV 2024, PT LXIV, 15122:294–310. Cham: Springer. https://doi.org/10.1007/978-3-031-73039-9_17.
- Chicago author-date (all authors)
- Lee, Wei-Yu, Martin Dimitrievski, David Van Hamme, Jan Aelterman, Ljubomir Jovanov, and Wilfried Philips. 2024. “CARB-Net : Camera-Assisted Radar-Based Network for Vulnerable Road User Detection.” In COMPUTER VISION - ECCV 2024, PT LXIV, 15122:294–310. Cham: Springer. doi:10.1007/978-3-031-73039-9_17.
- Vancouver
- 1.Lee W-Y, Dimitrievski M, Van Hamme D, Aelterman J, Jovanov L, Philips W. CARB-Net : Camera-Assisted Radar-Based Network for vulnerable road user detection. In: COMPUTER VISION - ECCV 2024, PT LXIV. Cham: Springer; 2024. p. 294–310.
- IEEE
- [1]W.-Y. Lee, M. Dimitrievski, D. Van Hamme, J. Aelterman, L. Jovanov, and W. Philips, “CARB-Net : Camera-Assisted Radar-Based Network for vulnerable road user detection,” in COMPUTER VISION - ECCV 2024, PT LXIV, Milan, Italy, 2024, vol. 15122, pp. 294–310.
@inproceedings{01JAMKQGC0SKM8RJ0VQAEPDBK0,
abstract = {{Ensuring a reliable perception of vulnerable road users is crucial for safe autonomous driving. Radar stands out as an appealing sensor choice due to its resilience in adverse weather, cost-effectiveness, depth sensing capabilities, and established role in adaptive cruise control. Nevertheless, radar's limited angular resolution poses challenges in object recognition, especially in distinguishing targets in close proximity. To tackle this limitation, we present the Camera-Assisted RadarBased Network (CARB-Net), a novel and efficient framework that merges the angular accuracy of a camera with the robustness and depth sensing capabilities of radar. We integrate camera detection information through a ground plane feed-forward array, entangling it with the early stages of a radar-based detection network. Furthermore, we introduce a unique context learning approach to ensure graceful degradation in situations of poor radar Doppler information or unfavorable camera viewing conditions. Experimental validations on public and our proposed datasets, along with benchmark comparisons, showcase CARBNet's superiority, boasting up to a 12% improvement in mAP performance. A series of ablation studies further emphasize the efficacy of the CARB-Net architecture. Our proposed dataset is released at https://github.com/weiyulee/RadVRU.}},
author = {{Lee, Wei-Yu and Dimitrievski, Martin and Van Hamme, David and Aelterman, Jan and Jovanov, Ljubomir and Philips, Wilfried}},
booktitle = {{COMPUTER VISION - ECCV 2024, PT LXIV}},
isbn = {{9783031730382}},
issn = {{0302-9743}},
keywords = {{Micro-Doppler Signature,Vulnerable Road User Detection,AUTOMOTIVE RADAR}},
language = {{eng}},
location = {{Milan, Italy}},
pages = {{294--310}},
publisher = {{Springer}},
title = {{CARB-Net : Camera-Assisted Radar-Based Network for vulnerable road user detection}},
url = {{http://doi.org/10.1007/978-3-031-73039-9_17}},
volume = {{15122}},
year = {{2024}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: