Learning to fly

Xu, Jie; Du, Tao; Foshey, Michael; Li, Beichen; Zhu, Bo; Schulz, Adriana; Matusik, Wojciech

doi:10.1145/3306346.3322940

Cited by 31 publications

(17 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vision-based Object Manipulation. The learning of robotic control policy has been traditionally studied with low-dimensional state observations [9,10,11,12,13]. Recently, vision-based policies [14,15,16,17,18,19,20,21] have gained increasing attention since the high-dimensional visual sensory input provides more generalizable observation representation across tasks and is more accessible in real-world perception systems.…”

Section: Related Workmentioning

confidence: 99%

IFOR: Iterative Flow Minimization for Robotic Object Rearrangement

Goyal

Mousavian²,

Paxton³

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

For 3D object manipulation, methods that build an explicit 3D representation perform better than those relying only on camera images. But using explicit 3D representations like voxels comes at large computing cost, adversely affecting scalability. In this work, we propose RVT, a multi-view transformer for 3D manipulation that is both scalable and accurate. Some key features of RVT are an attention mechanism to aggregate information across views and re-rendering of the camera input from virtual views around the robot workspace. In simulations, we find that a single RVT model works well across 18 RLBench tasks with 249 task variations, achieving 26% higher relative success than the existing stateof-the-art method (PerAct). It also trains 36X faster than PerAct for achieving the same performance and achieves 2.3X the inference speed of PerAct. Further, RVT can perform a variety of manipulation tasks in the real world with just a few (∼10) demonstrations per task. Visual results, code, and trained model are provided at: https://robotic-view-transformer.github.io/.

show abstract

Section: Related Workmentioning

confidence: 99%

IFOR: Iterative Flow Minimization for Robotic Object Rearrangement

Goyal

Mousavian²,

Paxton³

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…Hence learning directly on real hardware with RL is a formidable challenge gaining much attention lately [13], [14], [15]. However, such approaches demand significant prior knowledge or computing power and have only been applied to relatively simple and stable platforms or when we can access accurate simulators [16], [17], [18].…”

Section: A a Brief Survey Of Quadruped Robot Controlmentioning

confidence: 99%

Zero-Shot Learning Via Recurrent Knowledge Transfer

Zhao

Sun

Hong³

et al. 2019

2019 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Zero-shot learning (ZSL) which aims to learn new concepts without any labeled training data is a promising solution to large-scale concept learning. Recently, many works implement zero-shot learning by transferring structural knowledge from the semantic embedding space to the image feature space. However, we observe that such direct knowledge transfer may suffer from the space shift problem in the form of the inconsistency of geometric structures in the training and testing spaces. To alleviate this problem, we propose a novel method which actualizes recurrent knowledge transfer (RecKT) between the two spaces. Specifically, we unite the two spaces into the joint embedding space in which unseen image data are missing. The proposed method provides a synthesis-refinement mechanism to learn the shared subspace structure (SSS) and synthesize missing data simultaneously in the joint embedding space. The synthesized unseen image data are utilized to construct the classifier for unseen classes. Experimental results show that our method outperforms the state-of-the-art on three popular datasets. The ablation experiment and visualization of the learning process illustrate how our method can alleviate the space shift problem. By product, our method provides a perspective to interpret the ZSL performance by implementing subspace clustering on the learned SSS.

show abstract

“…For multirotors, the motor layout directly changes the aerodynamic and hydrodynamic performance (Xu et al, 2019) and then affects the vehicle's robustness in unstructured environments. Naviator adopts a dual‐layer four‐axis aerial thruster to achieve continuous cross‐domain motion (Maia et al, 2015), but the aerial propeller has low propulsion efficiency and endurance when working underwater (Tan et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

“…For multirotors, the motor layout directly changes the aerodynamic and hydrodynamic performance (Xu et al, 2019) and then affects the vehicle's robustness in unstructured environments.…”

mentioning

confidence: 99%

Nezha‐IV: A hybrid aerial underwater vehicle in real ocean environments

Jin,

Bi,

Lyu

et al. 2023

Journal of Field Robotics

View full text Add to dashboard Cite

Hybrid aerial underwater vehicles (HAUVs) that are capable of autonomously operating in both the aerial and underwater environments with repeated rapid transitions have received great interest in recent years. Current HAUV prototypes cannot fully cope with the unstructured, harsh, and hazardous ocean environments considering the high‐pressure and the strong hydrodynamic perturbations. This work reports on the HAUV, Nezha‐IV, with all basic functions validated in a real ocean environment. Nezha‐IV has four aerial propellers and eight underwater thrusters. It is not only capable of maneuvering delicately during flight and underwater but can also perform dynamic transitions under rough sea conditions. We present the design, construction, control, and oceanic testing of Nezha‐IV and demonstrate reliability and robustness, especially during the transition phases. Performance estimation and results gathered from tests in the South China Sea show that Nezha‐IV can operate at various depths ranging from 0 to 50 m at a velocity of 1 kt for 22 min and hover in the air for 15 min or cruise at a velocity of 10 m/s for 7.2 km in 12 min. This project contributes to the field of HAUV with (i) a HAUV capable of dynamic locomotion with high stability in both media and during transitions; (ii) identification of challenges during water–air transition in harsh sea conditions, and a reliable transition strategy that helps the development of fully autonomous HAUVs; (iii) a modular, foldable, and impact‐resistant HAUV structure; (iv) vehicle's capability proved by extensive tests in dynamic ocean environment.

show abstract

Learning to fly

Cited by 31 publications

References 34 publications

IFOR: Iterative Flow Minimization for Robotic Object Rearrangement

IFOR: Iterative Flow Minimization for Robotic Object Rearrangement

Zero-Shot Learning Via Recurrent Knowledge Transfer

Nezha‐IV: A hybrid aerial underwater vehicle in real ocean environments

Contact Info

Product

Resources

About