This paper presents a concept for an end-to-end system to evaluate the effectiveness of Maritime Situational Awareness (MSA) assets. One such prototype system, the Maritime Surveillance Simulator (MSS), developed at the NATO Undersea Research Centre (NURC) is introduced. The MSS system is highly modular and includes a global traffic model for large vessel traffic, sensor coverage models, a maritime data fusion engine, and metrics for overall system effectiveness. This paper describes the prototype MSS system and suggests further applicability for such a concept.