Robotic Arm Combined with the Visual Images in a Transparent Object Recognition

Chao, Chun-Kuan; Wu, Shang-Ya; Yan, Zhi-Tong; Tsai, Min-Long; Hsu, Chia-Chi; Raihany, Ully; Peng, Cheng-Yu

doi:10.1109/is3c50286.2020.00040

Cited by 1 publication

(1 citation statement)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Transparent objects are studied in both computer graphics and computer vision such as segmentation [CHW18, KTR*20], recognition [MNSiT13, CWY*20] and 3D reconstruction [HWL17, LYC20]. In this work, we propose a system for new view synthesis (NVS) of transparent objects.…”

Section: Introductionmentioning

confidence: 99%

Novel View Synthesis Of Transparent Object From a Single Image

Zhou

Wang

2022

Computer Graphics Forum

View full text Add to dashboard Cite

We propose a method for converting a single image of a transparent object into multi-view photo that enables users observing the object from multiple new angles, without inputting any 3D shape. The complex light paths formed by refraction and reflection makes it challenging to compute the lighting effects of transparent objects from a new angle. We construct an encoder-decoder network for normal reconstruction and texture extraction, which enables synthesizing novel views of transparent object from a set of new views and new environment maps using only one RGB image. By simultaneously considering the optical transmission and perspective variation, our network learns the characteristics of optical transmission and the change of perspective as guidance to the conversion from RGB colours to surface normals. A texture extraction subnetwork is proposed to alleviate the contour loss phenomenon during normal map generation. We test our method using 3D objects within and without our training data, including real 3D objects that exists in our lab, and completely new environment maps that we take using our phones. The results show that our method performs better on view synthesis of transparent objects in complex scenes using only a single-view image.

show abstract