Deep Q-Learning for Dry Stacking Irregular Objects

Liu, Yifang; Shamsi, Seyed Mahdi; Fang, Le; Chen, Changyou; Napp, Nils

doi:10.1109/iros.2018.8593619

Cited by 13 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furrer et al (36) used gradient descent and a support polygon-based cost function to create stacked towers of up to four stones, whereas Liu et al (35) achieved stacks up to six stones, and single-layer walls up to four courses, with a combination of hierarchical heuristics adapted from dry stone masonry guidebooks: Both studies built physical structures at the desktop scale in an open loop using prescanned stones that were localized for grasping with a manipulatormounted RGB-D sensor. Deep Q-learning has been applied for learning stone assembly policies from physics simulations in 2D scenes (37) and as a model-free method that uses 2.5D representations of the scene-top and stone-underside to position stones with fixed orientations toward the construction of double-faced walls in simulation (38). One limitation of the latter approach is the lack of consideration of the top surface of the candidate stones: Stones with outward-sloping top surfaces jeopardize the stability of subsequent stones (39,40), severely limiting the heights that can be achieved with the method.…”

Section: Robotic Construction With Raw and Reclaimed Materialsmentioning

confidence: 99%

A framework for robotic excavation and dry stone construction using on-site materials

Johns,

Wermelinger,

Mascaro

et al. 2023

Sci. Robot.

View full text Add to dashboard Cite

Automated building processes that enable efficient in situ resource utilization can facilitate construction in remote locations while simultaneously offering a carbon-reducing alternative to commonplace building practices. Toward these ends, we present a robotic construction pipeline that is capable of planning and building freeform stone walls and landscapes from highly heterogeneous local materials using a robotic excavator equipped with a shovel and gripper. Our system learns from real and simulated data to facilitate the online detection and segmentation of stone instances in spatial maps, enabling robotic grasping and textured 3D scanning of individual stones and rubble elements. Given a limited inventory of these digitized stones, our geometric planning algorithm uses a combination of constrained registration and signed-distance-field classification to determine how these should be positioned toward the formation of stable and explicitly shaped structures. We present a holistic approach for the robotic manipulation of complex objects toward dry stone construction and use the same hardware and mapping to facilitate autonomous terrain-shaping on a single construction site. Our process is demonstrated with the construction of a freestanding stone wall (10 meters by 1.7 meters by 4 meters) and a permanent retaining wall (65.5 meters by 1.8 meters by 6 meters) that is integrated with robotically contoured terraces (665 square meters). The work illustrates the potential of autonomous heavy construction vehicles to build adaptively with highly irregular, abundant, and sustainable materials that require little to no transportation and preprocessing.

show abstract

Section: Robotic Construction With Raw and Reclaimed Materialsmentioning

confidence: 99%

A framework for robotic excavation and dry stone construction using on-site materials

Johns,

Wermelinger,

Mascaro

et al. 2023

Sci. Robot.

View full text Add to dashboard Cite

show abstract

“…Later work by Li et al [61] in the same environment shows that reinforcement learning without demonstrations can learn to stack the cubes from state. Our stacking task is also related to other manipulation tasks, such as dry stacking [62,63] where rocks of irregular shapes must be stacked to form a wall, but these methods do not address. However, while these methods deal with high-level planning and goal understanding, our benchmark task requires dealing with low-level contact dynamics and perception to make real object stacking possible with strategies that emerge from RL training in simulation.…”

Section: F Additional Related Workmentioning

confidence: 99%

Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes

Lee¹,

Devin²,

Zhou³

et al. 2021

Preprint

View full text Add to dashboard Cite

We study the problem of robotic stacking with objects of complex geometry. We propose a challenging and diverse set of such objects that was carefully designed to require strategies beyond a simple "pick-and-place" solution. Our method is a reinforcement learning (RL) approach combined with visionbased interactive policy distillation and simulation-to-reality transfer. Our learned policies can efficiently handle multiple object combinations in the real world and exhibit a large variety of stacking skills. In a large experimental study, we investigate what choices matter for learning such general vision-based agents in simulation, and what affects optimal transfer to the real robot. We then leverage data collected by such policies and improve upon them with offline RL. A video and a blog post of our work are provided as supplementary material. 3

show abstract

Process Learning of Robot Fabric Manipulation Based on Composite Reward Functions

Zheng

et al. 2021

Intelligent Robotics and Applications

View full text Add to dashboard Cite

Deep Q-Learning for Dry Stacking Irregular Objects

Cited by 13 publications

References 15 publications

A framework for robotic excavation and dry stone construction using on-site materials

A framework for robotic excavation and dry stone construction using on-site materials

Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes

Process Learning of Robot Fabric Manipulation Based on Composite Reward Functions

Contact Info

Product

Resources

About