Nighttime three-dimensional target detection of driverless vehicles based on multi-view channel fusion network

WANG Yulan;SUN Shaoyuan;LIU Zhiyi;BU Defei

  • Sponsored by:

    Editor-In-Chief:

    ISSN 1002-2082

     
  • Hosted By:

    Published By: Journal of Applied Optics

    CN 61-1171/O4

Journal of Applied Optics ›› 2020, Vol. 41 ›› Issue (2) : 296-301. DOI: 10.5768/JAO202041.0202002

Nighttime three-dimensional target detection of driverless vehicles based on multi-view channel fusion network

  • WANG Yulan1,2, SUN Shaoyuan1,2, LIU Zhiyi1,2, BU Defei1,2
Author information +
History +

Abstract

In order to improve the ability of driverless vehicles to identify objects in the surrounding environment at night, a three-dimensional target detection method for driverless vehicles based on multi-view channel fusion network was proposed. The idea of multi-sensor fusion was adopted, and the target detection was carried out by adding laser radar point cloud on the basis of infrared image. By encoding the laser radar point cloud into a bird’s-eye view form and a front view form, as well as forming a multi-view channel with the infrared night vision image, the information of each channel was fused and complemented, thereby improving the ability of the driverless vehicle to recognize surrounding objects at night. The infrared image and the laser radar point cloud were used as the input of the network. The network accurately detected the position of the target and the category by the feature extraction layer, the regional suggestion layer and the channel fusion layer. The method can improve the object recognition ability of driverless vehicles at night, the accuracy rate in the laboratory test data reaches 90%, and the time consumption of 0.43 fps also basically meets the practical application requirements.

Key words

infrared image / lidar point cloud / three-dimensional target detection / multi-view channel

Cite this article

Download Citations
WANG Yulan, SUN Shaoyuan, LIU Zhiyi, BU Defei. Nighttime three-dimensional target detection of driverless vehicles based on multi-view channel fusion network. Journal of Applied Optics. 2020, 41(2): 296-301 https://doi.org/10.5768/JAO202041.0202002

References

陈琪. 面向无人驾驶的环境感知技术研究[J]. 科技经济导刊,2018,26(26):77.
高凯珺, 孙韶媛, 姚广顺, 等. 基于深度学习的无人车夜视图像语义分割[J]. 应用光学,2017,38(3):421-428.
许路, 赵海涛, 孙韶媛. 基于深层卷积神经网络的单目红外图像深度估计[J]. 光学学报,2016,36(7):0715002.
姚广顺, 孙韶媛, 方建安, 等. 基于红外与雷达的夜间无人车场景深度估计[J]. 激光与光电子学进展,2017,54(12):164-170.
谌彤童. 三维激光雷达在自主车环境感知中的应用研究[D]. 长沙: 国防科学技术大学, 2011.
刘清. 基于激光雷达的三维目标检测[D]. 武汉: 华中科技大学, 2011.
KIESER D. Driverless vehicles and their impact on your portfolio[J]. Equity,2018,32(1):4.
CHEN Qi. Research on environment-aware tech-nology for unmanned driving[J]. Science and Te-chnology Economic Guide,2018,26(26):77.
GAO Kaijun, SUN Shaoyuan, YAO Guangshun, et al. Semantic segmentation of night vision images for unmanned vehicles based on deep learning[J]. Journal of Applied Optics,2017,38(3):421-428.
XU Lu, ZHAO Haitao, SUN Shaoyuan. Deep esti-mation of monocular infrared image based on deep convolutional neural network[J]. Acta Optica Sinica,2016,36(7):0715002.
YAO Guangshun, SUN Shaoyuan, FANG Jian'an, et al. Infrared and radar based depth estimation of night unmanned vehicle scene[J]. Laser & Optoelectronics Progress,2017,54(12):164-170.
ZHAN Tongtong. Application research of 3D Li-dar in autonomous vehicle environment perception[D]. ChangSha: National University of Defense Technology, 2011.
LIU Qing. Laser radar based 3D target detection[D]. WuHan: Huazhong University of S

21

Accesses

0

Citation

Detail

Sections
Recommended

/