Using Web Crawling for Automated Country Risk Assessment

Abstract

The latest and most accurate host country information is crucial for international market selection and bidding decisions. In today's digital world, this information can be timely gathered from websites on the Internet, the globally connected network system facilitating worldwide access to a wide range of information resources through a massive collection of private, public, academic, and government networks. Within construction companies, the traditional risk assessment process is generally based on searching for country information on the web and necessitates human input, which is subjective by nature and prone to misinterpretation and errors. With the help of automation, the traditional way of identifying and assessing country risks can change for the better. Better, latest, and most accurate data collection with web crawling algorithms, which are used for automated browsing, may be possible for country risk assessment with less human involvement, minimizing human errors and saving effort and time. Studies that explored the automated collection of information from the Internet for country risk assessment in construction are limited in the literature. In this paper, we assert that the web crawling technique can help search for data from the web and can be used to automatically develop country risk registers. Within this context, first of all, the web crawling algorithms will be explained, and how they can be used to automate the risk assessment process will be discussed. Then, a demonstrative example of an international construction company carrying out a country risk assessment process while preparing a bid for a highway project in Serbia will be depicted. The benefits of web crawling for automated country risk assessment will be discussed as well as its challenges.