Realistic Indoor Path Visualization with Real-Time Obstacle Avoidance in Augmented Reality

Within the project of “Realistic Indoor Path Visualization” the research group around Hannes Kaufmann focused on guiding people in indoor environments by augmenting the real world with virtual content. To achieve this, it was necessary to take advantage of real time data from sensors like a video camera and a depth sensor. The developed concept in this project is implemented for the Microsoft HoloLens but can be ported to any other AR device.

In order to address the challenge or indoor AR navigation, a novel algorithm for calculating a path towards the desired target was created. As testing environment, a complex indoor environment was considered. During our research the Field of View Path “FOVPath” algorithm was invented. It is a path planning algorithm which is able to consider walkable pathways, dynamic obstacles, the current user position and more importantly the current viewing direction of the user. The algorithm is able to calculate a path which is shown in a plausible way. In addition to that, it also reacts to the used display device, since the display area of AR devices usually only covers a fraction of the user’s field of view. Therefore, it is necessary to understand the user’s viewpoint, otherwise the user always has to search for the AR content in the real world.

In addition to the development of the “FOVPath” algorithm, different static and dynamic visualizations have been implemented. Since tracking in an indoor environment cannot be guaranteed at all times, a method called “VisMatchGuide” was designed. This method addresses the visual recognition capabilities of the user. The operator is getting an image, which is a composition of the virtual path and the real image of the environment. With that the user should be able to do a visual match with the real world for continuing along the path. Within the project we also implemented a reliable detection of obstacles in real time and a light source estimator in order to enhance the visual quality. For a realistic representation of AR content, the project also included an estimation for real light sources by analyzing RGB-D images. Our evaluations show good performance of the “FOVPath” implementation in a real indoor environment. The proposed methods can lead to usable applications for AR indoor navigation in future.