In order to realize more natural and various motions like humans, humanlike musculoskeletal tendon-driven humanoids have been studied. Especially, it is very challenging to design musculoskeletal body structure which consists of complicated bones, redundant powerful and flexible muscles, and large number of distributed sensors. In addition, it is very challenging to reveal humanlike intelligence to manage these complicated musculoskeletal body structure. This paper sums up life-sized musculoskeletal humanoids Kenta, Kotaro, Kenzoh and Kenshiro which we have developed so far, and describes key technologies to develop and control these robots

This paper aims to provide a specific example of how OpenAI's ChatGPT can be used in a few-shot setting to convert natural language instructions into a sequence of executable robot actions (Fig. 1). Generating programs for robots from natural language instructions is an attractive goal, but the practical application using ChatGPT is still in its early stages, and there is no established methodology yet. Here, we have designed easy-to-customize input prompts for ChatGPT that meet common requirements in many practical applications, including: 1) easy integration with robot execution systems or visual recognition programs, 2) applicability to various environments, and 3) the ability to provide long-step instructions while minimizing the impact of ChatGPT's token limit. Specifically, the prompts encourage ChatGPT to 1) output a sequence of predefined robot actions with explanations in a readable JSON format, 2) represent the operating environment in a formalized style, and 3) infer and output the updated state of the operating environment as the result of each operation, which will be input with the next instruction to allow ChatGPT to work based solely on the memory of the latest operations. Through experiments, we confirmed that the proposed prompts allow ChatGPT to act in accordance with the requirements in various environments. Additionally, we observed that ChatGPT's conversational ability allows users to adjust its output with natural language feedback, which is crucial for developing an application that is both safe and robust while providing a user-friendly interface. Users can easily customize the prompts as templates. The contribution of this paper is to provide and publish the prompts, which are generic enough to be easily modified to fit the requirements of each experimenter, thereby providing practical knowledge to the robotics research community. Our prompts and source code for using them are open-source and publicly available at https://github.com/microsoft/ChatGPT-Robot-Manipulation-Prompts. Fig. 1. This paper shows practical prompts for ChatGPT to generate for translating a sequences of executable robot actions from multi-step human instructions in various environments.

Many works in robot teaching either focus on teaching a high-level abstract knowledge such as task constraints, or low-level concrete knowledge such as the motion for accomplishing a task. However, we show that both high-level and low-level knowledge is required for teaching a complex task sequence such as opening and holding a fridge with one arm while reaching inside with the other. In this paper, we propose a body role division approach, which maps both high-level task goals and low-level motion obtained through human demonstration, to robots of various configurations. The method is inspired by facts on human body motion, and uses a body structural analogy to decompose a robot's body configuration into different roles: body parts that are dominant for achieving a demonstrated motion, and body parts that are substitutional for adjusting the motion to achieve an instructed task goal. Our results show that our method scales to robots of different number of arm links, and that both high and low level knowledge is mapped to achieve a multi-step dual arm manipulation task. In addition, our results indicate that when either the high or low level knowledge of the task is missing, or when mapping is done without the role division, a robot fails to open a fridge door or is not able to navigate its footprint appropriately for an upcoming task. We show that such results not only apply to human-shaped robots with two link arms, but to robots with less degrees of freedom such as a one link armed robot.

No abstract