A generic testing framework for agent-based simulation models

Gürcan, Önder; Dikenelli, Oğuz; Bernon, Carole

doi:10.1057/jos.2012.26

Cited by 14 publications

(12 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These numbers are then used in validation-suite tests. Similar to Collier and Ozik (2013) and Onggo et al (2014), Gurcan et al (2013) advocate the use of an automatic testing tool for ABS model verification and validation. However, they adopted a different approach.…”

Section: Test-driven Simulation Modellingmentioning

confidence: 92%

“…Modellers are forced to think about how their model is going to be verified and validated, even before they begin to develop it. The second advantage is that TDSM can be implemented using various unit-testing tools, or incorporated into a framework such as the one proposed by Gurcan et al (2013), Niazzi et al (2009) and Thiele et al (2014). To demonstrate how the proposed validation technique can be used in practice, we will show a few examples in which parts of the agent-based simulation software called MASSIM (MAritime Search SIMulation) are verified and validated using TDSM.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Test-driven simulation modelling: A case study using agent-based maritime search-operation simulation

Onggo

Karataş

2016

European Journal of Operational Research

View full text Add to dashboard Cite

Model verification and validation (V&V) is one of the most important activities in simulation modelling. Model validation is especially challenging for agent-based simulation (ABS). Techniques that can help to improve V&V in simulation modelling are needed. This paper proposes a V&V technique called Test-Driven Simulation Modelling (TDSM) which applies techniques from TestDriven Development in software engineering to simulation modelling. The main principle in TDSM is that a unit test for a simulation model has to be specified before the simulation model is implemented. Hence, TDSM explicitly embeds V&V in simulation modelling. We use a case study in maritime search operations to demonstrate how TDSM can be used in practice. Maritime search operations (and search operations in general) are one of the classic applications of Operational Research (OR). Hence, we can use analytical models from the vast search theory literature for unit tests in TDSM. The results show that TDSM is a useful technique in the verification and validation of simulation models, especially ABS models. This paper also shows that ABS can offer an alternative modelling approach in the analysis of maritime search operations. INTRODUCTIONAgent-Based Simulation (ABS) has become one of the commonly used tools to model and understand complex and nonlinear systems (North and Macal 2007). ABS provides a controlled environment for systematic experimentation using a simulation model that is formed from a set of interacting agents. There is no consensus on the definition of an agent in the ABS literature (Macal and North 2010). Instead, we have observed a spectrum of complexity in its definition. At one extreme, an ABS model is formed by a set of agents with a set of simple attributes (such as speed and detection range) and simple behaviours (such as move and rescue). At the other extreme, an ABS model can be composed of a set of agents with complex attributes (such as memory and bounded rationality) and complex abilities (such as planning and learning). However, most researchers agree that an agent is an autonomous entity (i.e. it makes independent decisions without any central control), has a set of objectives and interacts with other agents and its environment. One of the most important activities in ABS is model validation. There are similarities between ABS and Discrete-Event Simulation (DES). Both are able to represent stochastic dynamic systems and can track individuals' states throughout their lifecycles in the model. Hence, a number of validation techniques proposed for DES are also suitable for ABS, such as face validity, operational validity, white-box validation and black-box validation. Kleijnen (1995), Balci (1995 and Sargent (2013) provide a list of validation techniques in the context of stochastic dynamic simulation which are applicable to DES and ABS. All good simulation textbooks have at least one chapter that discusses validation techniques. Law (2014, chapter 5) and Banks et al. (2010, chapter 10) discuss various techniques...

show abstract

Section: Test-driven Simulation Modellingmentioning

confidence: 92%

Section: Introductionmentioning

confidence: 99%

Test-driven simulation modelling: A case study using agent-based maritime search-operation simulation

Onggo

Karataş

2016

European Journal of Operational Research

View full text Add to dashboard Cite

show abstract

“…There are some efforts [13,14,15,16,17], but these studies do not directly deal with model testing processes and focus on late validation and verification. As well, there are few proposed model testing frameworks to conduct validation and verification throughout the model testing process [15,18,19]. Among them, [18] proposed an integrated and automated testing framework, but unfortunately this framework not easy to use for noncomputer scientists.…”

Section: Introductionmentioning

confidence: 99%

“…As well, there are few proposed model testing frameworks to conduct validation and verification throughout the model testing process [15,18,19]. Among them, [18] proposed an integrated and automated testing framework, but unfortunately this framework not easy to use for noncomputer scientists.…”

Section: Introductionmentioning

confidence: 99%

“…Towards this objective, we took the generic testing framework proposed by Gürcan et al [18] and improved it one step further by taking into account the requirements of testing frameworks for ABMS. Previously, testing requirements for ABMS are defined and testing levels of ABMS that can be subject of the model testing process are clarified in our previous studies [18,22]. We also revise our multi-level testing categorization by keeping in sight the requirements of agent-based simulation testing frameworks.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation