No abstract

This paper proposes a novel remote monitoring system using a head mounted display (HMD). By using an HMD, a high degree of immersion and the sense of reality is provided to the user. Further, it becomes more convenient to move the camera position while controlling the robot arm on which the camera is mounted through the coordinates obtained from the HMD sensors. The proposed system performs two tasks. Initially, in order to render images in the HMD, the camera captures input images, and the PC connected to the robot arm sends the image to the PC connected to the HMD in real-time after which the image is rendered in the HMD. For the real-time monitoring, we increased the frame per second by reducing the data size. In order to reduce the data size, we define region of interest which is the region a user can see. Then, region of interest of the image is cropped and the resolution of entire image is degraded. Therefore, two images composed of cropped image and degraded image are transmitted and merged into an image. In this way, we can reduce the data size and provide the user with monitoring the image of the original quality. Next, using the inertial measurement unit sensor and base station sensors built into the HMD, the user controls the displacement of the camera during rotation and translation by controlling the robot arm by the user's own motion. The PC connected to the HMD transmits the motion coordinates of the HMD acquired from the sensor into the PC connected to the robot arm. To control the translation of the camera, we define a coordinate system that represents the y-axis as running from top to bottom, and the x-axis as running from the front to the back in three-dimensional space. Thus, each step motor along each axis, which controls the robot arm, has its movement controlled by supplying appropriate values of angular degrees. Through the above process, the proposed method implements a remote monitoring system with five degrees of freedom. In the experiment, we measured the data transmission delay time and the displacement error due to the robot arm's motion control to determine whether the system is suitable for a real-time remote monitoring system. Experimental results show that the proposed system can be optimally operated with a frame rate of 15 frames per second and a group of pixel resolutions of 640×640 of the cropped image and 240×120 of the degraded image. In addition, the average error rate of the robot arm's displacement was 6.45% when the camera position was controlled through the robot arm. INDEX TERMS Head mounted display, real-time control, remote monitoring system, robot arm.

Image outpainting is a very intriguing problem as the outside of a given image can be continuously filled by considering as the context of the image. This task has two main challenges. The first is to maintain the spatial consistency in contents of generated regions and the original input. The second is to generate a high-quality large image with a small amount of adjacent information. Conventional image outpainting methods generate inconsistent, blurry, and repeated pixels. To alleviate the difficulty of an outpainting problem, we propose a novel image outpainting method using bidirectional boundary region rearrangement. We rearrange the image to benefit from the image inpainting task by reflecting more directional information. The bidirectional boundary region rearrangement enables the generation of the missing region using bidirectional information similar to that of the image inpainting task, thereby generating the higher quality than the conventional methods using unidirectional information. Moreover, we use the edge map generator that considers images as original input with structural information and hallucinates the edges of unknown regions to generate the image. Our proposed method is compared with other state-of-the-art outpainting and inpainting methods both qualitatively and quantitatively. We further compared and evaluated them using BRISQUE, one of the No-Reference image quality assessment (IQA) metrics, to evaluate the naturalness of the output. The experimental results demonstrate that our method outperforms other methods and generates new images with 360°panoramic characteristics.Figure 1. Main concept of the proposed method. We rearrange the outer area of the image into its inner area. This enables us to consider bidirectional information, and it can generate a high-quality natural image that is superior to images produced by conventional methods.