Visual support system for remote control by adaptive ROI selection of monitoring robot

Abstract

A visual assistance system has become attractive as a technique to improve the efficiency and stability of remote control. While an operator controls a working robot, another autonomous monitoring robot evaluates a suitable viewpoint to observe the work site, and dynamically moves to the optimal viewpoint for monitoring. Choosing the observation region (ROI: region of interest) is equivalent to deciding the action of the following autonomous monitoring system. Therefore, we focus on ROI detection in our visual support system. We propose an ROI selection method to identify the most suitable observation point and interobject relations. The monitoring robot detects a gestalt of the scene in order to identify the relations between objects. Such an adaptive ROI in real time improves the efficiency of the remote control. The experimental results indicate the effectiveness of the proposed system in terms of execution time and number of errors.

Best Viewpoints for External Robots or Sensors Assisting Other Robots

“…for teleoperating an excavator in a hooking task. Results from the experiments in [101,102,103] also suggest the existence of regions of adjacent viewpoints of similar rating (measured using com- Manifolds are conceptually similar to aspect graphs [133]. Nodes on an aspect graph represent distinct views of an object based on the structure of an object.…”

“…There were three classes of deliberation. The first was based on the position of the objects of interest [42,101,102,103]. The robotic visual assistant selected a view to keep all the objects of interests (e.g., manipulated objects, robot, obstacles) in its field of view while reasoning about the composition of the objects in the view.…”

“…Samejima [102,103] used two assistant UGVs to autonomously provide external views to the operator of a primary UGV in a domain agnostic scenario. The only criterion was to include objects deemed relevant in the view.…”

“…The second most common metric was an error rate that was used by 16 out of 32 studies. The error rate was defined as either the number of collisions [101,103,102,94,105,106,92,93,42,95] or was task-specific (error distance in object placing [79], the classification accuracy of task outcome prediction [107], number of object drops during manipulation [95], or mistakes in REACH-ABILITY judgment [38,39,40,41]). The third was psychological metrics used by 14 out of 32 studies [82,98,104,81,92,93,38,39,40,41,42,89,95,100].…”

“…Those were measured by a post-completion questionnaire and assessed perceived mental workload, frustration, confidence, perceived task difficulty, perceived view helpfulness, perceived effectiveness, perceived level of task accomplishment, or perceived level of accuracy. The fourth metric was a success rate used by 4 out of 32 studies and measured as the ratio of successfully completed tasks to unsuccessfully completed tasks [98,103], quality of the final result [95], or task-specific [89]. The fifth metric was visual servoing or tracking performance used by 4 out of 32 studies [91,87,96,88].…”