With the steady growth of the amount of real-time data while drilling, operational decision-making is becoming both better informed and more complex.Therefore, as no human brain has the capacity to interpret and integrate all decision-relevant information from the data, the adoption of advanced algorithms is required not only for data interpretation but also for decision optimization itself. However, the advantages of the automatic decision-making are hard to quantify.The main contribution of this paper is an experiment in which we compare the decision skills of geosteering experts with those of an automatic decision support system in a fully controlled synthetic environment. The implementation of the system, hereafter called DSS-1, is presented in our earlier work [Alyaev et al. "A decision support system for multi-target geosteering." Journal of Petroleum Science and Engineering 183 (2019)]. For the current study we have developed an easy-to-use web-based platform which can visualize and update uncertainties in a 2D geological model. The platform has both user and application interfaces (GUI and API) allowing us to put human participants and DSS-1 into a similar environment and conditions.