Despite considerable progress in the development of rapid evaluation methods for physics-based reservoir model simulators there still exists a significant gap in acceleration and accuracy needed to enable complex optimization methods, including Monte Carlo and Reinforcement Learning. The latter techniques bear a great potential to improve existing workflows and create new ones for a variety of applications, including field development planning. Building on latest developments in modern deep learning technology, this paper describes an end-to-end deep surrogate model capable of modeling field and individual-well production rates given arbitrary sequences of actions (schedules) including varying well lo-cations, controls and completions. We focus on generalization properties of the surrogate model which is trained given a certain number of simulations. We study its spatial and time interpolation and extrapolation properties using the SPE9 case, followed by a validation on a large-scale real field. Our results indicate that the surrogate model achieves acceleration rates of about 15000x and 40000x for the SPE9 and the real field, respectively, incurring relative error ranging between 2% and 4% in the interpolation case, and between 5% and 12% in the various spacial and time extrapolation cases. These results provide concrete measures of the efficacy of the deep surrogate model as an enabling technology for the development of optimization techniques previously out of reach due to computational complexity.
What is the next best drilling decision to make in a Field Development Plan (FDP)? This is the key question we address in this work. A FDP consists of a sequence of decisions. Each action we take affects the reservoir and conditions any future decision. The novelty of our proposed approach is the consideration of the sequential nature of the decisions through the framework of Dynamic Programming (DP) and Reinforcement Learning (RL). In this framework, each scheduled drilling decision depends on the observations acquired between drillings. This methodology allows moving the focus from a static Field Development Plan optimization to a more dynamic framework that we call Field Development Policy Optimization. In addition to the formulation of this new framework, we have applied this methodology to optimize the development of a real oil and gas field. We model the FDP optimization problem under subsurface uncertainty as a Partially Observable Markov Decision Process (POMDP) and solve it through a RL algorithm in order to find an optimal drilling policy. Our methodology works for a general reservoir with a given set of geological model representing. To speed up the learning process we utilize a trained Deep Recurrent Neural Network (RNN) to approximate the reservoir simulator flows, which are subsequently used to compute the economic performance of a drilling policy through its discounted cash flows. The RNN is trained and tested on a set of reservoir simulator runs over randomly sampled realizations of our reservoir model, well location, type and control sequences of a drilling plan. From all the possible decisions involved in a FDP, we focus here only on finding optimal adaptive well drilling schedules (locations of vertical wells and well types). The RL agent learns the best drilling schedule policy by generating simulated episodes of experience and iteratively improving the policy using a Q value function approximated by a neural network trained across episodes. The final solution consists of an adaptive Field Development Plan yielding the highest expected Net Present Value (NPV), computed within a given, available time budget. It specifies an adaptive drilling schedule of producer and injector, well locations and well controls, as a function of the information obtained at each drilling step. The methodology has been applied to an actual reservoir for infill well location decisions. In this case, our objective is finding the best well placement and well type for the next producer and injector wells, as well as the optimization of the control schedule for new and preexisting wells in the reservoir. Our results show the learning progress of our RL algorithm until finding the optimal drilling plan. The robustness of the solution is evaluated across the best-trained policies. Methodology and results have been validated using a brute force sampling approach. Both RL and brute force approaches were possible due to our fast-to-compute RNN approximation of the reservoir simulator. This work represents, to our knowledge, the first application of an end-to-end AI workflow for Field Development Policy Evaluation in real fields, based on Reinforcement Learning and Deep Learning. The proposed methodology puts together an optimal field evaluation in planning and a surveillance workflow for a reactive decision-making.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.