Understanding manipulation in video

Brand, Matthew

doi:10.1109/afgr.1996.557249

Cited by 38 publications

(20 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the 'what is being learned' area, most imitation learning research focuses on learning assembly/pick-and-place operations (Brand, 1997), (Ehrenmann, 2002), (Kang, 1991), (Kuniyoshi, 1994), (Ogata, 1994), (Paul, 1996), (Tung, 1995). Such research generally looks at an input data stream, and attempts to segment the stream to identify actions performed by a human hand.…”

Section: Related Workmentioning

confidence: 99%

Imitation Learning of Whole-Body Grasps

Hsiao¹,

Lozano-Pérez²

2006

2006 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

Humans often learn to manipulate objects by observing other people. In much the same way, robots can use imitation learning to pick up useful skills. A system is demonstrated here for using imitation learning to teach a robot to grasp objects using both hand and whole-body grasps, which use the arms and torso as well as hands. Demonstration grasp trajectories are created by teleoperating a simulated robot to pick up simulated objects, and stored as sequences of keyframes in which contacts with the object are gained or lost. When presented with a new object, the system compares it against the objects in a stored database to pick a demonstrated grasp used on a similar object. Both objects are modeled as a combination of primitives-boxes, cylinders, and spheres-and the primitives for each object are grouped into 'functional groups' that geometrically match parts of the new object with similar parts of the demonstration object. These functional groups are then used to map contact points from the demonstration object to the new object, and the resulting adapted keyframes are adjusted and checked for feasibility. Finally, a trajectory is found that moves among the keyframes in the adapted grasp sequence, and the full trajectory is tested for feasibility by executing it in the simulation. The system successfully uses this method to pick up 92 out of 100 randomly generated test objects in simulation.

show abstract

Section: Related Workmentioning

confidence: 99%

Imitation Learning of Whole-Body Grasps

Hsiao¹,

Lozano-Pérez²

2006

2006 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

show abstract

“…Other recent attempts to provide an analysis of video in restricted domains include the work of Mann et al [15] and Siskind et al [19] who propose methods for analyzing the physical interactions between objects in a video sequence, and that of Brand [6] who looks at understanding human actions in video for the purpose of video summarization. Kollnig et al [13] have defined a vocabulary of motion verbs that are used to analyze the behavior of cars in video sequences of traffic scenes.…”

Section: Related Workmentioning

confidence: 99%

Summarization of videotaped presentations: automatic analysis of motion and gesture

Black

Minneman

et al. 1998

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Abstract-This paper presents an automatic system for analyzing and annotating video sequences of technical talks. Our method uses a robust motion estimation technique to detect key frames and segment the video sequence into subsequences containing a single overhead slide. The subsequences are stabilized to remove motion that occurs when the speaker adjusts their slides. Any changes remaining between frames in the stabilized sequences may be due to speaker gestures such as pointing or writing, and we use active contours to automatically track these potential gestures. Given the constrained domain, we define a simple set of actions that can be recognized based on the active contour shape and motion. The recognized actions provide an annotation of the sequence that can be used to access a condensed version of the talk from a Web page.

show abstract

“…In 1996 Brand created a blob-oriented 2D vision system that used 6 handcoded networks similar to HMMs to recognize actions Brand called "touching", "putting", "getting", "adding", and "removing", in highly constrained video of human activity [2]. In 2000 Brand and Kettnaker described work on a system that automatically learned HMMs (the states, transitions, and parameter values) from a similar 2D blob oriented input from a well-positione stationary desk camera in an office [3].…”

Section: Machines That Watch From Afarmentioning

confidence: 99%

Duo: A Human/Wearable Hybrid for Learning About Common Manipulate Objects

Kemp¹

2002

View full text Add to dashboard Cite

Abstract. Humanoid robots would benefit from a better understanding of common manipulable objects and the human behaviors associated with them. Duo is a human/wearable hybrid that is designed to learn about this important domain of human intelligence by interacting with natural manipulable objects in unconstrained environments. Duo's wearable Al system measures the kinematic configuration of the human's head, torso and dominant arm, while watching the workspace of the human's hand through a head-mounted camera. Duo also requests helpful actions from the human through speech via headphones. This paper presents results on an initial set of behaviors for Duo which lead to high-quality segmentations of common manipulable objects in unconstrained human environments. In Duo, the wearable AI system essentially subsumes the abilities of its cooperative human partner by sharing the human's sensory input and directing a portion of the human's actions. Together, the cooperative human and the wearable Al system can be thought of as constituting a new kind of humanoid robot that complements more traditional, wholly synthetic humanoid robots by allowing researchers to circumvent some of the currently unsolved problems in the field, from dextrous object manipulation to unrestricted mobility.

show abstract

Understanding manipulation in video

Cited by 38 publications

References 2 publications

Imitation Learning of Whole-Body Grasps

Imitation Learning of Whole-Body Grasps

Summarization of videotaped presentations: automatic analysis of motion and gesture

Duo: A Human/Wearable Hybrid for Learning About Common Manipulate Objects

Contact Info

Product

Resources

About