Deep reinforcement learning for predictive aircraft maintenance using probabilistic Remaining-Useful-Life prognostics

Lee, Juseong; Mitici, Mihaela

doi:10.1016/j.ress.2022.108908

Cited by 75 publications

(28 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, reinforcement learning methods have driven impressive advances in artificial intelligence in recent years, surpassing human performance in many domains [22][23][24][25][26][27]. More recently, some researchers have begun to use reinforcement learning to model collective motion in a learning way [28][29][30][31].…”

Section: Related Workmentioning

confidence: 99%

Modeling Collective Behavior for Fish School With Deep Q-Networks

et al. 2023

View full text Add to dashboard Cite

Modeling collective behavior is a way to better understand the mechanisms that govern collective animal behaviors. Traditional rule-based modeling methods rely heavily on human prior knowledge and may not provide a proper explanation of the phenomenon of collective behaviors. This paper proposes a Deep Q-Networks (DQN)-based modeling method for fish school. Firstly, an individual's state (continuous value) is expressed by the angle between its direction and the average direction of its perceived neighbors. An individual's action is represented with discretized turning angle. Then, the reward function is constructed with the change in the number of neighbors. And finally, the neural network structure is constructed to represent the Q-value function and is trained by the DQN algorithm. The proposed approach is tested in two scenarios: single-learner and multi-learner. Results show that in both scenarios the proposed method can gradually converge and finally obtain a model that can produce collective behavior. On this basis, this paper also deeply analyzes the learned model from the perspectives of average order parameter and collective behavior patterns. It verifies that the behavior pattern generated by the learned model is a highly ordered collective behavior. In addition, we make a comparison between our proposed approach and the Q-Learning algorithm. The results show that our approach not only has a stronger ability to express policy and is better at handling continuous states but also has a more stable learning performance in training. INDEX TERMSCollective behavior, collective behavior model, Deep Q-Networks (DQN), fish school

show abstract

Section: Related Workmentioning

confidence: 99%

Modeling Collective Behavior for Fish School With Deep Q-Networks

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Generally, the existing approaches for RUL estimation can be categorized into three main groups: model-based approaches [1]- [6], data-driven approaches [7]- [27] and hybrid approaches [28], [29]. Model-based approaches can precisely estimate the RUL if the degradation process of physical systems is accurately modeled.…”

Section: Introductionmentioning

confidence: 99%

“…In addition to DL, deep reinforcement learning (DRL) is also introduced to the field of prognostics. Lee and Mitici [27] proposed a framework integrating RUL prognostics into predictive maintenance planning. In this framework, the RUL distribution is firstly estimated by CNN combining with Monte Carlo dropout.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Remaining Useful Life Estimation in Prognostics Using Deep Reinforcement Learning

Zhao

Wang

et al. 2023

IEEE Access

View full text Add to dashboard Cite

In modern industrial systems, condition-based maintenance (CBM) has been wildly adopted as an efficient maintenance strategy. Prognostics, as a key enabler of CBM, involves the kernel task of estimating the remaining useful life (RUL) for engineered systems. Much research in recent years has focused on developing new machine learning (ML) based approaches for RUL estimation. A variety of ML algorithms have been employed in these approaches. However, there was no research on applying deep reinforcement learning (DRL) to RUL estimation. To fill this research gap, a novel DRL based prognostic approach is proposed for RUL estimation in this paper. In the proposed approach, the conventional RUL estimation task is first formulated into a Markov decision process (MDP) model. Then an advanced DRL algorithm is employed to learn the optimal RUL estimation policy from this MDP environment. The effectiveness and superiority of the proposed approach are demonstrated through a case study on turbofan engines in C-MAPSS dataset. Compared to other approaches, the proposed approach obtains superior performance on all four sub-datasets of C-MAPSS dataset. What is more, on the most complicated subdatasets FD002 and FD004, the RMSE metric is improved by 14.4% and 7.81%, and the score metric is improved by 3.7% and 48.79%, respectively.

show abstract

“…It has heard a lot about RL since it beat the thenreigning world champion in a game of Go. In fact, RL has been successfully applied to many use cases [23]- [25] due to its remarkable ability to interact with various real-world environments. For the issue of dropout-layer inconsistencies, we adopt a method called Regularized Dropout (R-Drop) [26] to maintain the consistency among dropout layers.…”

Section: Introductionmentioning

confidence: 99%

Performance Improvement on Traditional Chinese Task-Oriented Dialogue Systems With Reinforcement Learning and Regularized Dropout Technique

Sheu

Wu³

2023

IEEE Access

View full text Add to dashboard Cite

The development of conversational voice assistant applications has been in full swing around the world. This paper aims to develop traditional Chinese multi-domain task-oriented dialogue (TOD) systems. It is typically implemented using pipeline approach, where submodules are optimized independently, resulting in inconsistencies with each other. Instead, this paper implements end-to-end multi-domain TOD models using pre-trained deep neural networks (DNNs). This allows us to integrate all the submodules into one single DNN model to solve the inconsistencies. Data shortages are common in conversational natural language processing (NLP) tasks using DNN models. In this regard, dropout regularization has been widely used to improve overfitting caused by insufficient training dataset. However, the randomness it introduces leads to non-negligible discrepancies between training and inference. On the other hand, pre-trained language models have successfully provided effective regularization for NLP tasks. An inherent disadvantage is that fine-tuning the pre-trained language model suffers from exposure bias and loss-evaluation mismatch. To this end, we propose a reinforcement learning (RL) approach to address both issues. Furthermore, we adopt a method called regularized dropout (R-Drop) to improve the inconsistency in dropout layers of DNNs. Experimental results show that both our proposed RL approach and the R-Drop technique can significantly improve the joint target accuracy (JGA) score and combined score of traditional Chinese TOD system in tasks of dialogue state tracking (DST) and end-to-end sentence prediction, respectively.INDEX TERMS NLP, regularized dropout, reinforcement learning, task-oriented dialogue

show abstract

Deep reinforcement learning for predictive aircraft maintenance using probabilistic Remaining-Useful-Life prognostics

Cited by 75 publications

References 44 publications

Modeling Collective Behavior for Fish School With Deep Q-Networks

Modeling Collective Behavior for Fish School With Deep Q-Networks

Remaining Useful Life Estimation in Prognostics Using Deep Reinforcement Learning

Performance Improvement on Traditional Chinese Task-Oriented Dialogue Systems With Reinforcement Learning and Regularized Dropout Technique

Contact Info

Product

Resources

About