Increasing the use of evidence in policy making means strengthening capacity on both the supply and demand sides of evidence production. However, little experience of strengthening the capacity of policy makers in low- and middle- income countries has been published to date. We describe the experiences of five projects (in Bangladesh, Gambia, India and Nigeria), where collaborative teams of researchers and policy makers/policy influencers worked to strengthen policy maker capacity to increase the use of evidence in policy. Activities were focused on three (interlinked) levels of capacity building: individual, organizational and, occasionally, institutional. Interventions included increasing access to research/data, promoting frequent interactions between researchers and members of the policy communities, and increasing the receptivity towards research/data in policy making or policy-implementing organizations. Teams were successful in building the capacity of individuals to access, understand and use evidence/data. Strengthening organizational capacity generally involved support to infrastructure (e.g. through information technology resources) and was also deemed to be successful. There was less appetite to address the need to strengthen institutional capacity—although this was acknowledged to be fundamental to promoting sustainable use of evidence, it was also recognized as requiring resources, legitimacy and regulatory support from policy makers. Evaluation across the three spheres of capacity building was made more challenging by the lack of agreed upon evaluation frameworks. In this article, we propose a new framework for assessing the impact of capacity strengthening activities to promote the use of evidence/data in policy making. Our evaluation concluded that strengthening the capacity of individuals and organizations is an important but likely insufficient step in ensuring the use of evidence/data in policy-cycles. Sustainability of evidence-informed policy making requires strengthening institutional capacity, as well as understanding and addressing the political environment, and particularly the incentives facing policy makers that supports the use of evidence in policy cycles.