Human-Computer Interaction and Human-Building Interaction

Vision-Based Mobile Indoor Assistive Navigation Aid for Blind People

Bing Li, Juan Pablo Munoz, Xuejian Rong, Qingtian Chen, Jizhong Xiao, Yingli Tian, Aries Arditi, Mohammed Yousuf

Published 2019

Thumbnail for Vision-Based Mobile Indoor Assistive Navigation Aid for Blind People
📄 Paper
This paper presents a new holistic vision-based mobile assistive navigation system to help blind and visually impaired people with indoor independent travel. The system detects dynamic obstacles and adjusts path planning in real-time to improve navigation safety. First, we develop an indoor map editor to parse geometric information from architectural models and generate a semantic map consisting of a global 2D traversable grid map layer and context-aware layers. By leveraging the visual positioning service (VPS) within the Google Tango device, we design a map alignment algorithm to bridge the visual area description file (ADF) and semantic map to achieve semantic localization. Using the on-board RGB-D camera, we develop an efficient obstacle detection and avoidance approach based on a time-stamped map Kalman filter (TSM-KF) algorithm. A multi-modal human-machine interface (HMI) is designed with speech-audio interaction and robust haptic interaction through an electronic SmartCane. Finally, field experiments by blindfolded and blind subjects demonstrate that the proposed system provides an effective tool to help blind individuals with indoor navigation and wayfinding.

Abstract

This paper presents a new holistic vision-based mobile assistive navigation system to help blind and visually impaired people with indoor independent travel. The system detects dynamic obstacles and adjusts path planning in real-time to improve navigation safety. First, we develop an indoor map editor to parse geometric information from architectural models and generate a semantic map consisting of a global 2D traversable grid map layer and context-aware layers. By leveraging the visual positioning service (VPS) within the Google Tango device, we design a map alignment algorithm to bridge the visual area description file (ADF) and semantic map to achieve semantic localization. Using the on-board RGB-D camera, we develop an efficient obstacle detection and avoidance approach based on a time-stamped map Kalman filter (TSM-KF) algorithm. A multi-modal human-machine interface (HMI) is designed with speech-audio interaction and robust haptic interaction through an electronic SmartCane. Finally, field experiments by blindfolded and blind subjects demonstrate that the proposed system provides an effective tool to help blind individuals with indoor navigation and wayfinding.

Topics

Cross-Modal Intelligence and Multimodal IntegrationHCI-HBIXRAI automated accessibilityXRAI for AECXRAI inclusive design

Cite this paper

@ARTICLE{0caf18e5,
  author={Li, Bing and Muñoz, Juan Pablo and Rong, Xuejian and Chen, Qingtian and Xiao, Jizhong and Tian, Yingli and Arditi, Aries and Yousuf, Mohammed},
  journal={IEEE Transactions on Mobile Computing},
  title={Vision-Based Mobile Indoor Assistive Navigation Aid for Blind People},
  year={2019},
  volume={18},
  number={3},
  pages={702-714},
  keywords={Navigation;Collision avoidance;Semantics;Cameras;Path planning;Real-time systems;Google;Indoor assistive navigation;semantic maps;obstacle avoidance;Google Tango device;blind and visually impaired people},
  doi={10.1109/TMC.2018.2842751}}