discuss interoperability testing at scale. In 2002, Microsoft began the difficult process of verifying much of the technical documentation for its Windows communication protocols. The undertaking came about as a consequence of a consent decree Microsoft entered into with the U.S. Department of Justice and several state attorneys general that called for the company to make available certain client-server communication protocols for third-party licensees.A series of RFC-like technical documents were then written for the relevant Windows client-server and server-server communication protocols, but to ensure interoperability Microsoft needed to verify the accuracy and completeness of those documents. From the start, it was clear this wouldn't be a typical QA (quality assurance) project. First and foremost, a team would be required to test documentation, not software, which is an inversion of the normal QA process; and the documentation in question was extensive, consisting of more than 250 documents-30,000 pages in all. In addition, the compliance deadlines were tight. To succeed, the Microsoft team would have to find an efficient testing methodology, identify the appropriate technology, and train an army of testers-all within a very short period of time.This case study considers how the team arrived at an approach to that enormous testing challenge. More specifically, it focuses on one of the testing methodologies used-model-based testing-and the primary challenges that have emerged in adopting that approach for a very large-scale project. Two lead engineers from the Microsoft team and an engineer who played a role in reviewing the Microsoft effort tell the story. Now with Google, Wolfgang Grieskamp at the time of this project was part of Microsoft's Windows Server and Cloud Interoperability Group (Winterop), the group charged with testing Microsoft's protocol documentation and, more generally, with ensuring that Microsoft's platforms are interoperable with software from the world beyond Microsoft. Previously, Grieskamp was a researcher at Microsoft Research, where he was involved in efforts to develop model-based testing capabilities. Nico Kicillof, who worked with Grieskamp at Microsoft Research to develop a model-based testing tool called Spec Explorer, continues to guide testing efforts as part of the Winterop group. Bob Binder is an expert on matters related to the testing of communication protocols. He too has been involved with the Microsoft testing project, having served as a test methodology consultant who also reviewed work performed by teams of testers in China and India. For this case study, Binder spoke with Kicillof and Grieskamp regarding some of the key challenges they've faced over the course of their large-scale testing effort. BOB BINDER When you first got involved with the Winterop Team [the group responsible for driving the creation, publication, and QA of the Windows communication protocols], what were some of the key challenges?NICO KICILLOF The single greatest challenge was that we were faced...
