Abstract. In recent global business environments, collaborations among organisations raise an increased demand for swift establishment. Such collaborations are formed between organisations entering Virtual Organizations (VOs), crossing geographic borders and frequently without prior experience of the other partner's previous performance. In VOs, every participant risks engaging with partners who may exhibit unexpected fraudulent or otherwise untrusted behaviour. In order to cope with this risk, the STochastic REputation system (STORE) was designed to provide swift, automated decision support for selecting partner organisations in the early stages of the VO's formation. The contribution of this paper first consists of a multi-agent simulation framework design and implementation to evaluate the STORE reputation system. This framework is able to simulate dynamic agent behaviour, agents hereby representing organisations, and to capture the business context of different VO application scenarios. A configuration of agent classes is a powerful tool to obtain not only well or badly performing agents for simulation scenarios, but also agents which are specialized in particular VO application domains or even malicious agents, attacking the VO community. The second contribution comprises of STORE's evaluation in two simulation scenarios, set in the VO application domains of Collaborative Engineering and Ad-hoc Service provisioning. Besides the ability to clearly distinguish between agents of different classes according to their reputation, the results prove STORE's ability to take an agent's dynamic behaviour into account. The simulation results show, that STORE solves the difficult task of selecting the most trustworthy partner for a particular VO application domain from a set of honest agents that are specialized in a wide spread of VO application domains.