This paper addresses the problem of online inverse reinforcement learning for nonlinear systems with modeling uncertainties while in the presence of unknown disturbances. The developed approach observes state and input trajectories for an agent and identifies the unknown reward function online. Sub-optimality introduced in the observed trajectories by the unknown external disturbance is compensated for using a novel model-based inverse reinforcement learning approach. The observer estimates the external disturbances and uses the resulting estimates to learn the dynamic model of the demonstrator. The learned demonstrator model along with the observed suboptimal trajectories are used to implement inverse reinforcement learning. Theoretical guarantees are provided using Lyapunov theory and a simulation example is shown to demonstrate the effectiveness of the proposed technique.
Over the years, oil companies have continually looked for unique ways to advance technology in the drilling industry, with one of the significant achievements being horizontal drilling techniques. Horizontal wells allow for a more extended reach into the reservoir, resulting in more oil being extracted per well, however, these horizontal sections increase drilling distance and ultimately cost. These higher drilling costs increase the need for better optimization methods. Many researchers have analyzed theoretical ROP equations for optimization, though most have only incorporated constant drilling parameters in these equations. Since formation variables change with depth, so should the optimum drilling variables. Therefore, constant parameters waste both time and money, which could be immensely improved with the use of dynamic drilling parameters. The method presented herein, incorporates a Particle Swarm Optimization (PSO) algorithm to a rate of penetration (ROP) model in order to minimize the overall cost per foot of the well. This is achieved by allowing the PSO to find the best combination of drilling parameters, downhole weight on bit (DWOB) and revolutions per minute (RPM) of the drill bit, along with optimized pull depth and bit combinations. The application of this algorithm could be applied in a variety of ways including for operating oil and gas companies to plan for new wells or as an artificial intelligence component of a drilling simulator. A long term use could be as an autonomous driller constantly getting information updates and solving real time optimal solutions.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.