Leveraging a User Research Framework to Guide Research Investments: Windows Vista Case Study

Abstract

During the development of Windows Vista we had the opportunity to invest in new methods to understand user behavior. We leveraged standard usability methods to work on feature areas during development; however, we had to invent and adapt new approaches to measure holistic experiences. In this area user research methods are evolving, due to the integration of technologies and changes in the definition of a successful experience. While considering the methods that suited our needs, a user research framework was created. This helped us manage investments in research activities. The framework is organized along two dimensions: perspective and time. Perspective refers to the breadth of the experience being considered: 'narrow' defines a focus on an individual feature area or small product area, and 'broad' defines a focus on an integrated experience. Time can indicate either a product cycle or real time. For product cycle most of the research is spent on the evaluation of the designs of the features and experiences related to predicting user behavior for a particular release of a product, whereas real time is our research investment into understanding how products are used in the wild without our intervention. Each quadrant of the two-dimensional framework highlights different research methods and purposes. It's important to realize that the value of the framework comes from the integration of findings that provides a rich holistic picture of our users to ultimately guide product decisions. This paper describes some of the methods that were evolved and created during the development of Windows Vista and their relationship to the user research framework. The methods described in the paper include user experience score-carding, measurement of desirability, and the impact of the consumer adoption program. These methods continued to be used today in the development of Windows 7.