| dc.description.abstract | Operational decision-making during drilling for hydrocarbons or geothermal energy is challenging due to the complex nature of the process. Many of the times, these decisions have to be taken with incomplete information at hand, either because of uncertainties or errors in the measurements, limited data transmission rates or bandwidths, inaccurate modeling during planning phase, delays in processing and analyzing the information or simply due to unexpected situations encountered in the process. This is more evident in geosteering operations while drilling directionally, because a series of high-quality decisions regarding well-trajectory adjustment are required to be taken in timely manner to achieve optimal result. Lack of a systematic and transparent framework to clearly and quantitatively state measurable objectives, key underlying uncertainties, or relevance between underlying uncer-tainties and real-time information has led to the treatment of geosteering operations as more of an art, rather than science. Furthermore, these challenges encountered in geosteering when it is treated as an independent operation are multiplied when the operation is integrated into the broader framework of drilling operations and it becomes a daunting task for a driller on the surface, or a drilling engineer at the rig or at a remote location, to carry out the operations in an eÿcient manner. Automating the processes and incorporating autonomous systems into the workflow is a viable solution to this highly complex problem. In this work, machine learning techniques, in particular Reinforcement Learning has been used to develop autonomous geosteering systems and predictive analytics for wellbore cleaning and vibration mitigation problems. The problems are first solved in a modular fashion and a plan to integrate the sub-systems is proposed later.
In this work, I frame the geosteering operation as a sequential decision-making problem under uncertainty. Conventional procedures to automate the process are either model-based which require accurate modeling of the highly complex process, or not universal thus limiting applicability of the methods with freedom, or computationally expensive. The approach taken in this study utilize reinforcement learning techniques to develop solutions that is model-free, platform-independent, and are near-real-time. A physics-based real-time engine has been used to develop a drilling simulator using which the reinforcement learning models are trained. Traditional dynamic programming solutions are not tractable because of the continuous action space for steering, possible observations within each sequence, and the computational demands of simulating updated environment. Function approximators are used in a policy gradient approach utilizing an advanced technique of Proximal Policy Opti-mization to train a drilling agent. Two separate solutions for steering have been developed, one for accurately tracking a given wellbore trajectory under uncertainty, and the other to self-learn the trajectory and maximize contact with target zone.
A second set of solutions pertain to improving the performance of the drilling operations. A dysfunction library has been developed using supervised machine learning techniques to identify in real-time the type of dysfunction encountered downhole. The solutions are ex-tended beyond identification, into prediction of a particular type of dysfunction, Stickslip, ahead of time. The models were able to predict Stickslip 10 seconds and 30 seconds into the future and have been validation using field data. Reinforcement Learning has been used to optimize the hole cleaning process essentially minimizing cuttings bed height by changing surface parameters. These solutions are developed as independent sub-systems which could eventually be integrated into the reinforcement learning framework of autonomous geosteering. My main contributions in this study are 1) to formulate the drilling systems in sequential decision-making framework 2) implement reinforcement learning and other machine learning techniques, 3) develop solutions that are interpretable and practical and 4) develop a plan for an integrated optimization solution that could pave way for a fully autonomous downhole drilling system in the future. | |
