Objectives

This project aims at developing and integrating the required sensing, acting and interacting capabilities to have mobile robotic platforms be used to conduct virtual visits for remote consultations or assistance and allow vital signs monitoring.

A telepresence mobile robot is a remote-controlled, wheeled device with wireless Internet connectivity for bidirectional audio, video and data transmission. In health care, a clinician or a caregiver could assist seniors in their homes using a telepresence robot without having to travel to these locations. Many mobile telepresence robotic platforms have recently been introduced on the market, allowing mobile telecommunication and vital sign monitoring at reasonable cost. What is missing for making them effective remote telepresence systems for home care assistance are capabilities specifically needed to assist the remote operator in controlling the robot and perceiving the environment through the robot’s sensors. The objective is to minimize cognitive load and maximize situation awareness of the operator. This can be accomplished by presenting the robot’s environment using contextualized sensory data on the graphical user interface and by providing autonomous navigation and vital sign monitoring functions to the platform.

The developed software involves the integration and improvement of a cloud-based multi-streams video/audio/data telecommunication platform, an artificial audition system, an appearance-based mapping and localization navigation system to help operating the robot with a web-based or dedicated application.  As a proof of concept, we integrated our software and hardware to the Suitable Technologies beam+ platform, demonstrating the feasibility of adding these capabilities on a low cost, already available commercial robot.