Today, enhancement in sensing technology enables the use of multiple sensors to track human motion/activity precisely. Tracking human motion has various applications, such as fitness training, healthcare, rehabilitation, human-computer interaction, virtual reality, and activity recognition. Therefore, the fusion of multiple sensors creates new opportunities to develop and improve an existing system. This paper proposes a pose-tracking system by fusing multiple three-dimensional (3D) light detection and ranging (lidar) and inertial measurement unit (IMU) sensors. The initial step estimates the human skeletal parameters proportional to the target user’s height by extracting the point cloud from lidars. Next, IMUs are used to capture the orientation of each skeleton segment and estimate the respective joint positions. In the final stage, the displacement drift in the position is corrected by fusing the data from both sensors in real time. The installation setup is relatively effortless, flexible for sensor locations, and delivers results comparable to the state-of-the-art pose-tracking system. We evaluated the proposed system regarding its accuracy in the user’s height estimation, full-body joint position estimation, and reconstruction of the 3D avatar. We used a publicly available dataset for the experimental evaluation wherever possible. The results reveal that the accuracy of height and the position estimation is well within an acceptable range of ±3–5 cm. The reconstruction of the motion based on the publicly available dataset and our data is precise and realistic.

Understanding and differentiating subtle human motion over time as sequential data is challenging. We propose Motion-sphere, which is a novel trajectory-based visualization technique, to represent human motion on a unit sphere. Motion-sphere adopts a two-fold approach for human motion visualization, namely a three-dimensional (3D) avatar to reconstruct the target motion and an interactive 3D unit sphere, that enables users to perceive subtle human motion as swing trajectories and color-coded miniature 3D models for twist. This also allows for the simultaneous visual comparison of two motions. Therefore, the technique is applicable in a wide range of applications, including rehabilitation, choreography, and physical fitness training. The current work validates the effectiveness of the proposed work with a user study in comparison with existing motion visualization methods. Our study’s findings show that Motion-sphere is informative in terms of quantifying the swing and twist movements. The Motion-sphere is validated in threefold ways: validation of motion reconstruction on the avatar, accuracy of swing, twist, and speed visualization, and the usability and learnability of the Motion-sphere. Multiple range of motions from an online open database are selectively chosen, such that all joint segments are covered. In all fronts, Motion-sphere fares well. Visualization on the 3D unit sphere and the reconstructed 3D avatar make it intuitive to understand the nature of human motion.

Human pose estimation and tracking in real-time from multi-sensor systems is essential for many applications. Combining multiple heterogeneous sensors increases opportunities to improve human motion tracking. Using only a single sensor type, e.g., inertial sensors, human pose estimation accuracy is affected by sensor drift over longer periods. This paper proposes a human motion tracking system using lidar and inertial sensors to estimate 3D human pose in real-time. Human motion tracking includes human detection and estimation of height, skeletal parameters, position, and orientation by fusing lidar and inertial sensor data. Finally, the estimated data are reconstructed on a virtual 3D avatar. The proposed human pose tracking system was developed using open-source platform APIs. Experimental results verified the proposed human position tracking accuracy in real-time and were in good agreement with current multi-sensor systems.

No abstract

No abstract

Synthesizing realistic human motion data using a real-time motion capture system in a controlled environment is a critical challenge. In addition, effectively manipulating the existing motion data is another primary concern and using such modified data in human motion analysis and activity recognition systems are prone to errors. This paper presents a simplistic and comprehensive system to effortlessly author, edit, and validate human motion data. The system enables a naive user to edit the existing motion data interactively using a humanoid model in a three-dimensional space, based on user-defined scenarios and synthesize numerous variations of the motion sequences. A modular concept of scenario-based sensed unit motion editing has been adopted to demonstrate the proposed system. We employed an efficient analytical kinematic and constraint solver to enforce the inherent body joint limits and external constraints while editing to synthesize complete and meaningful motion sequences. Furthermore, we substantiated the proposed sensed unit motion editing framework through a visual validation study using an open-source intuitive visualization tool called the Motion-Sphere. Finally, we compared the resultant synthesized motion against the real-time motion capture system data to verify the body segments' orientation and position accuracy deviations.

No abstract

Human action recognition (HAR) technology is receiving considerable attention in the field of human-computer interaction. We present a HAR system that works stably in real-world applications. In realworld applications, the HAR system needs to identify detailed actions for specific purposes, and the action data includes many variations. Accordingly, we conducted three experiments. First, we tested our recognition system's performance on the UTD-MHAD dataset. We compared our system's accuracy with results from previous research and confirmed that our system achieves a 91% average performance among recognition systems. Furthermore, we hypothesized the use of a HAR system to detect burglary. In the second experiment, we compared the existing benchmark data with our crime detection dataset. We recognized the test scenarios' data by using the recognition system trained by each dataset. The recognition system trained by our dataset achieved higher accuracy than the past benchmark dataset. The results show that the training data should contain detailed actions for a real application. In the third experiment, we tried to find the motion data type that stably recognizes action regardless of data variation. In a real application, the action data are changed by people. Thus, we introduced variations in the action data using the cross-subject protocol and moving area setting. We trained the recognition system using each position and angle data. In addition, we compared the accuracy of each system. We found that the angle format results in better accuracy because the angle data are beneficial for converting the action variation into a consistent pattern.