Wheelie – A HoloLens Driven Electrical Wheelchair

Equipped with specialized sensors and a custom holographic processing unit, the HoloLens enables us to go beyond the screen, blend in the digital world with the physical one to come up with more intuitive solutions to our everyday problems. We, at AIT GmbH, built a proof-of-concept, combining the spatial mapping capabilities of the HoloLens with an electrical wheelchair, creating an autonomous wheelchair. 

 

 

As demonstrated in the video above, the user is able to control the chair without using a joystick or directional speech commands, but by only supporting the position he wants to go to, either by an “air tap” or with a speech command. The HoloLens receives this input and based upon spatial mapping data it continuously receives about its surroundings, it controls and navigates the chair exactly to where the user intended.

 

How it all works

The HoloLens uses its specialized sensors to constantly build a 3D visualization of its surroundings. These data are represented as 3D geometry which are then fed into a navigation component that builds a navigation mesh based on the radius of the chair and height of the user. The resulting mesh is shown to the user as a blue floor, showing the possible traversable areas, on which the chair can drive.

 

When the user chooses his intended destination, the HoloLens checks if it lies on the generated traversable mesh and computes a path towards it. The HoloLens then controls the chair, rotating it and moving it along the generated path. The path is steadily updated, reflecting the changes in the environment as people move around and furniture changes place.

 

 

The Chair

Alber GmbH was our first choice for an electrical wheelchair collaborator. Their work on bringing joy through their wheelchair extensions resonated with our values and the modularity of their products allowed for external tools to extend their established soluions. An efix-35 was provided by Alber GmbH for the sake of this project.

 

Conclusion

The spatial mapping capabilities of the HoloLens were paired with a navigation algorithm to build a driverless wheelchair, allowing people to control their chair by only looking at their intended destination. As augmented reality continues to grow, more and more ways are being discovered for it to fill up more roles and become an integral part of our daily lives.

Stay tuned for future blogs about this project and our ventures in the augmented and virtual reality world.