This paper presents a convex optimization approach to control the density distribution of autonomous mobile agents with two control modes: ON and OFF. The main new characteristic distinguishing this model from standard Markov decision models is the existence of the ON control mode and its observed actions. When an agent is in the ON mode, it can measure the instantaneous outcome of one of the actions corresponding to the ON mode and decides whether it should take this action or not based on this new observation. If it does not take this action, it switches to the OFF mode where it transitions to the next state based on a predetermined set of transitional probabilities, without making any additional observations. In this decisionmaking model, each agent acts autonomously according to an ON/OFF decision policy, and the discrete probability distribution for the agent's state evolves according to a discrete-time Markov chain that is a linear function of the stochastic environment (i.e., transition probabilities) and the ON/OFF decision policy. The relevant policy synthesis is formulated as a convex optimization problem where hard safety and convergence constraints are imposed on the resulting Markov matrix. We first consider the case where the ON mode has a single action and the OFF mode has deterministic transitions (rather than stochastic) to demonstrate the model and the ideas behind our approach, which is then generalized to the case where ON mode has multiple actions and OFF mode has stochastic transitions.
Accessing difficult to reach hydrocarbon reservoirs while simultaneously reducing risk and increasing efficiency demonstrates a need for improved autonomous directional control of rotary steerable systems (RSS). The inherently uncertain drilling environment presents a challenge for control algorithms and human operators alike, where model mismatch can be significant and the parameters are time varying. Parameter estimation can improve the performance of steering controllers through model adaptation as well as provide valuable information to human operators. This paper proposes the use of a Markov Chain Monte Carlo (MCMC) based method to estimate time-varying model parameters in real-time using only measurements commonly obtained while drilling. The proposed method is evaluated on historical field data and the estimator's accuracy is quantified by prediction accuracy to achieve a mean absolute error of 0.68 degrees over 30 m. Next, the proposed method is used to adapt the model of a model predictive controller (MPC) and its performance is compared with a static MPC in a closed-loop simulation of a realistic drilling scenario. The results show the estimator improves tracking performance by 93.36%. Lastly, the utility of estimation for human-in-the-loop operation is explored through the design of an early warning system (EWS). The posterior distribution produced by MCMC is utilized in the EWS to predict the probability of undesirable future trajectories. By providing automatic alerts, the EWS serves as a safety mechanism and improves the operator's proficiency when monitoring several autonomously drilled wells.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.