Shear Wave Velocity Estimation Based on Deep-Q Network

Abstract

Geoacoustic inversion is important for seabed geotechnical applications. It can be formulated as a problem that seeks an optimal solution in a high-dimensional parameter space. The conventional inversion approach exploits optimization methods with a pre-defined search strategy whose hyperparameters need to be fine-tuned for a specific scenario. A framework based on the deep-Q network is proposed in this paper and the environment and agent configurations of the framework are specially defined for geoacoustic inversion. Unlike a conventional optimization method with a pre-defined search strategy, the proposed framework determines a flexible strategy by trial and error. The proposed framework is evaluated by two case studies for estimating the shear wave velocity profile. Its performance is compared with three global optimization methods commonly used in underwater geoacoustic inversion. The results demonstrate that the proposed framework performs the inversion more efficiently and accurately.

Shear Wave Velocity Estimation by Deep Reinforcement Learning: A Case Study

“…Please note that this section is a brief rephrasing of the DQN-based inversion framework proposed in our previous work. More details about the framework and the configuration of the environment and agent can be found in [10].…”

“…With the development of artificial intelligence and machine learning, deep reinforcement learning (DRL) is getting attention for its superiority in intelligent control and robotics, which can iteratively update the model by interacting with the environment to achieve good data fitting [9]. From this perspective, DRL can be intuitively applied to geoacoustic inversion by exploiting a DRL model instead of the conventional optimization method to guide the agent search in the parameter space, which can introduce a learnable search strategy and conduct inversion more efficiently [10].…”

“…The DQN-framework was proposed in our previous work and validated in simulation cases [10]. This paper aims to assess the inversion performance of the DQN-framework on real cases based on field data.…”

DQN-GNN-Based User Association Approach for Wireless Networks

“…where r is the immediate reward obtained after taking action a in state s,s is the next state, and θ − represents the parameters of a separate target network that are updated less frequently than the online network. The loss function used in DQN is the mean squared error (MSE) between the predicted Q-values and the target Q-values [22], given in Equation (3).…”

Determining the number of soil layers using the surface wave method based on decision tree and random forest

“…The common methods used in the inversion analysis of surface waves include iterative techniques such as least-squares fitting and direct search algorithms, which aim to find the parameters that best fit the experimental data. Recent developments in inversion analysis include the use of Levenberg-Marquardt method [6], genetic algorithm [7], neighborhood algorithm [8], simulated annealing [9], artificial bee colony algorithm [10], Monte Carlo search technique [11], deep-Q network search algorithm [12], and particle swarm optimization [13].…”