Understanding the cost of fitness evaluation for subset selection

Mengshoel, Ole J.; Flogard, Eirik Lund; Yu, Tong; Riege, Jon

doi:10.1145/3512290.3528689

Cited by 4 publications

(2 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, when simulations are used for AST, computational time and cost can be a serious problem [15], [23], [13], [10], [21]. In fact, the problem of fitness function evaluation cost, where "cost" may refer to computational cost, energy cost, engineering cost, or other costs, is a more general problem in AI [27], [9], [14], [20], [22]. Second, failure events other than collisions are of great interest.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Green Urban Mobility with Autonomous Electric Ferries: Studies of Simulated Maritime Collisions using Adaptive Stress Testing

Vatle,

Eriksen,

Mengshoel

2024

Linköping Electronic Conference Proceedings

View full text Add to dashboard Cite

With 90% of the world's goods transported by sea vessels, it is crucial to investigate their safety. This is increasingly important as autonomy is being introduced into sea vessels, which transport goods and people. To study the safety of an autonomous ferry's collision avoidance system, we consider the Adaptive Stress Testing (AST) method in this work. AST uses machine learning, specifically reinforcement learning, along with a simulation of a system under test---in our case, an autonomous and electric ferry---and its environment. Whether that simulation is fully or partially observable has implications for the integration into existing engineering workflows. The reason is that the fully observable simulation induces a more complex interface than the partially observable simulation, meaning that the engineers designing and implementing AST need to acquire and comprehend more potentially complex domain knowledge. This paper presents maritime adaptive stress testing (MAST) methods, using the world's first autonomous, electric ferry used to transport people as a case study. Using MAST in multiple scenarios, we demonstrate that AST can be productively utilized in the maritime domain. The demonstration scenarios stress test a maritime collision avoidance system known as Single Path Velocity Planner (SP-VP). Additionally, we consider how MAST can be implemented to test using both fully observable (gray box) and partially observable (black box) simulators. Consequently, we introduce the Gray-Box MAST (G-MAST) and Black-Box MAST (B-MAST) architectures, respectively. In simulation experiments, both architectures successfully identify an almost equal number of failure events. We discuss lessons learned about MAST including the experiences with both the Gray-Box and Black-Box approaches.

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

Green Urban Mobility with Autonomous Electric Ferries: Studies of Simulated Maritime Collisions using Adaptive Stress Testing

Vatle,

Eriksen,

Mengshoel

2024

Linköping Electronic Conference Proceedings

View full text Add to dashboard Cite

show abstract

“…Based on the results from the study and the feedback from the inspectors, it is likely that doing so could further increase inspection performance. The dataset used for the prototype does not have many features, so adding more features and using feature selection could be an option to optimize performance [13,26,24,25]. Another direction for future work is to take advantage of the transparency of the CBCBR method, and develop more explanation methods to promote and increase the inspectors' trust in the system.…”

Section: Discussionmentioning

confidence: 99%

Creating Explainable Dynamic Checklists via Machine Learning to Ensure Decent Working Environment for All: A Field Study with Labour Inspections

Flogard,

Mengshoel,

Theisen

et al. 2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

To address poor working conditions and promote United Nations’ sustainable development goal 8.8, “protect labour rights and promote safe working environments for all workers [...]”, government agencies around the world conduct labour inspections. To carry out these inspections, inspectors traditionally use paper-based checklists as a means to survey individual organisations for working environment violations. Currently, these checklists are created by domain experts, but recent research indicates that machine learning (ML) could be used to generate dynamic checklists to increase inspection efficiency. A drawback with the dynamic checklists is that they are complex and could be difficult to understand for inspectors. They have also never been field-tested. In this paper, we therefore propose user-oriented explanation methods for Context-aware Bayesian Case-Based Reasoning (CBCBR), which is the current state-of-art ML method for generating dynamic checklists. We also introduce a prototype of CBCBR and present a field study where we test it in real-world labour inspections. The results from the study indicate that using the explainable dynamic checklists increases the efficiency of the labour inspections, and inspectors also report that they find the checklists useful. The results also suggest that current ML evaluation methods, where model prediction performance is evaluated on existing data, may not fully reflect the real-world field performance of checklists.

show abstract

Comparing Metaheuristic Optimization Algorithms for Ambulance Allocation: An Experimental Simulation Study

Schjølberg,

Bekkevold,

Sánchez-Díaz

et al. 2023

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

Understanding the cost of fitness evaluation for subset selection

Cited by 4 publications

References 39 publications

Green Urban Mobility with Autonomous Electric Ferries: Studies of Simulated Maritime Collisions using Adaptive Stress Testing

Green Urban Mobility with Autonomous Electric Ferries: Studies of Simulated Maritime Collisions using Adaptive Stress Testing

Creating Explainable Dynamic Checklists via Machine Learning to Ensure Decent Working Environment for All: A Field Study with Labour Inspections

Comparing Metaheuristic Optimization Algorithms for Ambulance Allocation: An Experimental Simulation Study

Contact Info

Product

Resources

About