This paper proposes a task decomposition and dedicated reward-system-based reinforcement learning algorithm for the Pick-and-Place task, which is one of the high-level tasks of robot manipulators. The proposed method decomposes the Pick-and-Place task into three subtasks: two reaching tasks and one grasping task. One of the two reaching tasks is approaching the object, and the other is reaching the place position. These two reaching tasks are carried out using each optimal policy of the agents which are trained using Soft Actor-Critic (SAC). Different from the two reaching tasks, the grasping is implemented via simple logic which is easily designable but may result in improper gripping. To assist the grasping task properly, a dedicated reward system for approaching the object is designed through using individual axis-based weights. To verify the validity of the proposed method, wecarry out various experiments in the MuJoCo physics engine with the Robosuite framework. According to the simulation results of four trials, the robot manipulator picked up and released the object in the goal position with an average success rate of 93.2%.
Hallux valgus (HV), which is mainly caused by the wearing of narrow-width and high-heeled shoes, disrupts gait behavior because it deforms lower limb joints. There is limited information regarding the relationship between the foot disease HV and lower limb joints. Previous studies evaluating abnormal gait patterns caused by deformity used spatiotemporal parameters; however, they failed to characterize the overall gait dynamics. To address this issue, this study is aimed at characterizing the gait stability of patients with HV and examining the joints that are critically affected by HV. To assess complex gait dynamics, we quantified the potential changes in gait stability by using the maximum Lyapunov exponent (MLE). Angular displacements of the ankle, knee, and hip in the sagittal plane during walking were investigated to calculate the MLE for gait stability based on foot conditions (i.e., barefoot, flat shoes, and high heels). During walking, a large MLE (
P
<
0.05
) was noted for the knee joints of subjects with HV, relative to the other lower limb joints. HV appears to have the most critical effect on the knee joints during walking. Ankle movement exhibited higher dynamic stability than the other joint movements of the lower limb (
P
<
0.05
). The type of shoe used in the experiment showed no significant dependence with gait stability and joint movement (
P
>
0.05
). Quantitative assessments of dynamic stability using the MLE may help clinicians assess the overall gait dynamics of HV patients and other people suffering from gait disturbances.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.