Batch Reinforcement Learning of Feasible Trajectories in a Ship Maneuvering Simulator

Amendola, José; Tannuri, Eduardo A.; Cozman, Fábio Gagliardi; Reali, Anna

doi:10.5753/eniac.2018.4422

Cited by 3 publications

(2 citation statements)

References 15 publications

(10 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Channel navigation was handled as a path-following problem. In (AMENDOLA et al, 2018), the port channel adopted in the experiments was straight and no environment conditions were considered, the state space adopted was simpler and an algorithm called State space in path planning tasks carries information about the absolute position in a given scenario. The policy, in this case, is learned for that specific setup with very limited capacity of generalization.…”

Section: Literature Reviewmentioning

confidence: 99%

“…(RANDLØV; ALSTRØM, 1998) argued that negative values prevent the agent from going through states unnecessarily to accumulate positive rewards. Nonetheless, in previous work (AMENDOLA et al, 2018), reward functions with negative values for most state-space led the agent to quickly go for collisions in order to minimize overall negative punishment. As limited available space in a channel prevents the vessel from navigating away from the centerline, positive rewards are not an issue.…”

Section: Rewardmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement learning applied to vessel navigation in fast-time simulations.

Andrade¹

View full text Add to dashboard Cite

Fast-time simulations have been proven to be an essential tool for maritime engineering, not only in ship design but also by detecting critical situations and bottlenecks in projects of ports. However, such simulations are not performed by professional pilots and might become a complex task with results not so close to reality. Such issues can present an opportunity for introducing Reinforcement Learning methods in the maritime domain.This work proposes a Reinforcement Learning based solution which is able to automatically generate vessel trajectories in restricted waters under the effect of environment forces. The agent learns by interacting with the simulator and receiving reward signals.It also gives discrete commands in spaced time steps in order to emulate limitations of human piloting.The method evaluates the distributed version of two state-of-art Reinforcement Learning algorithms. It handles channel segments as separate episodes and includes curvature information for anticipating actions. Experiments were run considering realistic scenarios with narrow curved channels where wind and current incidence varies along the trajectory.The novelty of the work is the fact that the solution proposed requires no prior knowledge on dynamic models or predefined line paths to be followed by the ship. It may impact in fast-time simulations by requiring less human effort in trajectories generation. The method adopted keeps a simple representation and can be applied to any port channel configuration that respects local technical regulations.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Rewardmentioning

confidence: 99%

Reinforcement learning applied to vessel navigation in fast-time simulations.

Andrade¹

View full text Add to dashboard Cite

show abstract

A preliminary experiment combining marine robotics and citizenship engagement using imitation learning

et al. 2020

View full text Add to dashboard Cite

Navigation in Restricted Channels Under Environmental Conditions: Fast-Time Simulation by Asynchronous Deep Reinforcement Learning

et al. 2020

View full text Add to dashboard Cite

This paper proposes an efficient method, based on reinforcement learning, to be used as ship controller in fast-time simulators within restricted channels. The controller must operate the rudder in a realistic manner in both time and angle variation so as to approximate human piloting. The method is well suited to scenarios where no previous navigation data is available; it takes into account, during training, both the effect of environmental conditions and also curves in channels. We resort to an asynchronous distributed version of the reinforcement learning algorithm Deep Q Network (DQN), handling channel segments as separate episodes and including curvature information as context variables (thus moving away from most work in the literature). We tested our proposal in the channel of Porto Sudeste, in the southern Brazilian coast, with realistic environment scenarios where wind and current incidence varies along the channel. The method keeps a simple representation and can be applied to any port channel configuration that respects local technical regulations.

show abstract

Batch Reinforcement Learning of Feasible Trajectories in a Ship Maneuvering Simulator

Cited by 3 publications

References 15 publications

Reinforcement learning applied to vessel navigation in fast-time simulations.

Reinforcement learning applied to vessel navigation in fast-time simulations.

A preliminary experiment combining marine robotics and citizenship engagement using imitation learning

Navigation in Restricted Channels Under Environmental Conditions: Fast-Time Simulation by Asynchronous Deep Reinforcement Learning

Contact Info

Product

Resources

About