CNN-Based Obstacle Avoidance Using RGB-Depth Image Fusion

Chaymae El Mechal*, Najiba El Amrani El Idrissi, Mostefa Mesbah

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)


In the last few years, deep learning has attracted wide interest and achieved great success in many computer vision related applications, such as image classification, object detection, object tracking, pose estimation and action recognition. One specific application that can greatly benefit from the recent advance of deep learning is robot vision-based obstacle avoidance. Vision-based obstacle avoidance systems are mostly based on classification algorithms. Most of these algorithms use either color images or depth images as the main source of information. In this paper, the aim is to investigate whether using information extracted from both types of images simultaneously would give better performance than using each one separately. To do this, we chose the convolutional neural network (CNN) as the classifier and HSV-based method to achieve the fusion. We tested this approach using two widely used pre-trained CNN architectures, namely Resnet-50 and GoogLeNet using a dataset locally collected. The results indicate that the image fusion-based classification algorithm achieve a higher accuracy (91.3%) than the one based on depth images (80.4%) but lower than the one based on color images (93.7%). These results can be partly explained by the fact that the used classifiers were pre-trained using color image datasets.

Original languageEnglish
Title of host publicationWITS 2020 - Proceedings of the 6th International Conference on Wireless Technologies, Embedded, and Intelligent Systems
EditorsSaad Bennani, Younes Lakhrissi, Ghizlane Khaissidi, Anass Mansouri, Youness Khamlichi
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages10
ISBN (Print)9789813368927
Publication statusPublished - 2022
Event6th International Conference on Wireless Technologies, Embedded and Intelligent Systems, WITS 2020 - fez, Morocco
Duration: Oct 14 2020Oct 16 2020

Publication series

NameLecture Notes in Electrical Engineering
ISSN (Print)1876-1100
ISSN (Electronic)1876-1119


Conference6th International Conference on Wireless Technologies, Embedded and Intelligent Systems, WITS 2020


  • Convolutional neural network
  • Deep learning
  • Image fusion
  • Obstacle avoidance
  • Robot vision

ASJC Scopus subject areas

  • Industrial and Manufacturing Engineering


Dive into the research topics of 'CNN-Based Obstacle Avoidance Using RGB-Depth Image Fusion'. Together they form a unique fingerprint.

Cite this