Autonomous ships are expected to change water-based transport of both cargo and people, and large investments are being made internationally. There are many reasons for such transformation and interest, including shifting transport of goods from road to sea, reducing ship manning costs, reduced dangerous exposure for crew, and reduced environmental impact. Situational awareness (SA) systems and Autonomous navigation systems (ANS) are key elements of autonomous ships. Safe deployment of ANS will not be feasible based on real-life testing only. The assurance of autonomous ships and systems will require large-scale, systematic simulation-based testing in addition to assurance of the development process. DNV GL proposes to use a digital twin, that is a digital representation of key elements of the autonomous vessel as a key tool for the simulation-based testing, focusing on functional testing, failure tolerance, and performance aspects. The digital twin contains comprehensive mathematical models of the ship and its equipment, including all sensors and actuators. The complete simulationbased test system complementing the digital twin should consist of a virtual world to simulate environmental conditions, geographical information and interaction with other maritime traffic and obstacles. Finally, the test system must include a test management system that controls the simulations in the digital twin and the virtual world, generates test scenarios as well as evaluates the test scenario results. The scenario generation should automatically search for low system performance, and ultimately establish sufficient coverage of the possible scenario space. The test scenario evaluation should automatically consider safety, conformance to collision regulations at sea (COLREGs), and possibly the efficiency of the ship navigation. This paper presents a comprehensive prototype of a test system for ANS. Key topics will be simulation-based testing, interfacing between the simulator and ANS, cooperation with ANS manufacturers, dynamic test scenario generation and automatic assessment towards COLREGs.