Learning high-DOF reaching-and-grasping via dynamic representation of gripper-object interaction

She, Qijin; Hu, Ruizhen; Xu, Juzhan; Liu, Min; Xu, Kai; Huang, Hui

doi:10.1145/3528223.3530091

Cited by 63 publications

(19 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For physical recognition, we processed data from three sensors: magnetometer, gyroscope, and accelerometer [58][59][60][61]. Various statistical features were extracted.…”

Section: Feature Extraction For Physical Activitymentioning

confidence: 99%

Biosensor-Driven IoT Wearables for Accurate Body Motion Tracking and Localization

Almujally,

Khan,

Al Mudawi

et al. 2024

Sensors

View full text Add to dashboard Cite

The domain of human locomotion identification through smartphone sensors is witnessing rapid expansion within the realm of research. This domain boasts significant potential across various sectors, including healthcare, sports, security systems, home automation, and real-time location tracking. Despite the considerable volume of existing research, the greater portion of it has primarily concentrated on locomotion activities. Comparatively less emphasis has been placed on the recognition of human localization patterns. In the current study, we introduce a system by facilitating the recognition of both human physical and location-based patterns. This system utilizes the capabilities of smartphone sensors to achieve its objectives. Our goal is to develop a system that can accurately identify different human physical and localization activities, such as walking, running, jumping, indoor, and outdoor activities. To achieve this, we perform preprocessing on the raw sensor data using a Butterworth filter for inertial sensors and a Median Filter for Global Positioning System (GPS) and then applying Hamming windowing techniques to segment the filtered data. We then extract features from the raw inertial and GPS sensors and select relevant features using the variance threshold feature selection method. The extrasensory dataset exhibits an imbalanced number of samples for certain activities. To address this issue, the permutation-based data augmentation technique is employed. The augmented features are optimized using the Yeo–Johnson power transformation algorithm before being sent to a multi-layer perceptron for classification. We evaluate our system using the K-fold cross-validation technique. The datasets used in this study are the Extrasensory and Sussex Huawei Locomotion (SHL), which contain both physical and localization activities. Our experiments demonstrate that our system achieves high accuracy with 96% and 94% over Extrasensory and SHL in physical activities and 94% and 91% over Extrasensory and SHL in the location-based activities, outperforming previous state-of-the-art methods in recognizing both types of activities.

show abstract

“…For physical recognition, we processed data from three sensors: magnetometer, gyroscope, and accelerometer [58][59][60][61]. Various statistical features were extracted.…”

Section: Feature Extraction For Physical Activitymentioning

confidence: 99%

Biosensor-Driven IoT Wearables for Accurate Body Motion Tracking and Localization

Almujally,

Khan,

Al Mudawi

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Interaction Representation : Various methods have been proposed to extract the interaction feature between two parts, such as a hand and a object or a human body and the surrounding environment [ZWK14,SHX*22]. Zhao et al [ZWK14] propose to extract the Interaction Bisector Surface (IBS) between two objects using a geometry‐based method, and use this feature for the classification and retrieval of 3D objects.…”

Section: Related Workmentioning

confidence: 99%

“…Zhao et al [ZWK14] propose to extract the Interaction Bisector Surface (IBS) between two objects using a geometry‐based method, and use this feature for the classification and retrieval of 3D objects. She et al [SHX*22] use IBS to represent the gripper‐object interaction between gripper and object to solve the high‐DOF reaching‐and‐grasping problem. The study of interaction feature between the human body and the surrounding scene has been explored in the field of 3D human reconstruction and generation [ZHN*20, ZZM*20, HGT*21].…”

Section: Related Workmentioning

confidence: 99%

Reconstructing 3D Human Pose from RGB‐D Data with Occlusions

Dang,

Zhao,

Zhang

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

We propose a new method to reconstruct the 3D human body from RGB‐D images with occlusions. The foremost challenge is the incompleteness of the RGB‐D data due to occlusions between the body and the environment, leading to implausible reconstructions that suffer from severe human‐scene penetration. To reconstruct a semantically and physically plausible human body, we propose to reduce the solution space based on scene information and prior knowledge. Our key idea is to constrain the solution space of the human body by considering the occluded body parts and visible body parts separately: modeling all plausible poses where the occluded body parts do not penetrate the scene, and constraining the visible body parts using depth data. Specifically, the first component is realized by a neural network that estimates the candidate region named the “free zone”, a region carved out of the open space within which it is safe to search for poses of the invisible body parts without concern for penetration. The second component constrains the visible body parts using the “truncated shadow volume” of the scanned body point cloud. Furthermore, we propose to use a volume matching strategy, which yields better performance than surface matching, to match the human body with the confined region. We conducted experiments on the PROX dataset, and the results demonstrate that our method produces more accurate and plausible results compared with other methods.

show abstract

“…These findings attest to the potential of Smart IMU, GPS, Audio, and Ambient Sensors in precisely identifying and classifying a range of human activities ( Gioanni et al, 2016 ). Beyond exploring deep learning techniques, this research paper introduces a hybrid system ( She et al, 2022 ; Liang et al, 2018 ; Liu et al, 2022d ; Vrskova et al, 2023 ; Surek et al, 2023 ) that blends machine learning and deep learning features. By capitalizing on the strengths of both paradigms, the hybrid system further sharpens activity recognition, signaling a promising avenue for future research and development.…”

Section: Introductionmentioning

confidence: 99%

Robust human locomotion and localization activity recognition over multisensory

Khan,

Alonazi,

Abdelhaq

et al. 2024

Front. Physiol.

View full text Add to dashboard Cite

Human activity recognition (HAR) plays a pivotal role in various domains, including healthcare, sports, robotics, and security. With the growing popularity of wearable devices, particularly Inertial Measurement Units (IMUs) and Ambient sensors, researchers and engineers have sought to take advantage of these advances to accurately and efficiently detect and classify human activities. This research paper presents an advanced methodology for human activity and localization recognition, utilizing smartphone IMU, Ambient, GPS, and Audio sensor data from two public benchmark datasets: the Opportunity dataset and the Extrasensory dataset. The Opportunity dataset was collected from 12 subjects participating in a range of daily activities, and it captures data from various body-worn and object-associated sensors. The Extrasensory dataset features data from 60 participants, including thousands of data samples from smartphone and smartwatch sensors, labeled with a wide array of human activities. Our study incorporates novel feature extraction techniques for signal, GPS, and audio sensor data. Specifically, for localization, GPS, audio, and IMU sensors are utilized, while IMU and Ambient sensors are employed for locomotion activity recognition. To achieve accurate activity classification, state-of-the-art deep learning techniques, such as convolutional neural networks (CNN) and long short-term memory (LSTM), have been explored. For indoor/outdoor activities, CNNs are applied, while LSTMs are utilized for locomotion activity recognition. The proposed system has been evaluated using the k-fold cross-validation method, achieving accuracy rates of 97% and 89% for locomotion activity over the Opportunity and Extrasensory datasets, respectively, and 96% for indoor/outdoor activity over the Extrasensory dataset. These results highlight the efficiency of our methodology in accurately detecting various human activities, showing its potential for real-world applications. Moreover, the research paper introduces a hybrid system that combines machine learning and deep learning features, enhancing activity recognition performance by leveraging the strengths of both approaches.

show abstract

Learning high-DOF reaching-and-grasping via dynamic representation of gripper-object interaction

Cited by 63 publications

References 51 publications

Biosensor-Driven IoT Wearables for Accurate Body Motion Tracking and Localization

Biosensor-Driven IoT Wearables for Accurate Body Motion Tracking and Localization

Reconstructing 3D Human Pose from RGB‐D Data with Occlusions

Robust human locomotion and localization activity recognition over multisensory

Contact Info

Product

Resources

About