Assessing the credibility of evidence in complex, socio‐scientific problems is of paramount importance. However, there is little discussion in the science education literature on this topic and on how students can be supported in developing such skills. In this article, we describe an instructional design framework, which we call the Credibility Assessment Framework, to scaffold high school students' collaborative construction of evidence‐based decisions and their assessment of the credibility of evidence. The framework was employed for the design of a web‐based reflective inquiry environment on a socio‐scientific issue, and was enacted with 11th grade students. The article describes the components of the Credibility Assessment Framework and provides the details and results of an empirical study illustrating this framework in practice. The results are presented in the form of a case study of how 11th grade students investigated and evaluated scientific data relating to the cultivation of genetically modified plants. Multiple kinds of data were collected, including pre‐ and post‐tests of students' conceptual understanding and their skills in assessing the credibility of evidence, and videotapes of students' collaborative inquiry sessions. The analysis of the pre‐ and post‐tests on students' conceptual understanding of Biotechnology and their skills in assessing the credibility of evidence revealed statistically significant learning gains. Students' work in task‐related artifacts and the analysis of two groups' videotaped discussions showed that students became sensitive to credibility criteria, questioned the sources of data and correctly identified sources of low, moderate, and high credibility. Implications for designers and educators regarding the application of this framework are discussed. © 2011 Wiley Periodicals, Inc. J Res Sci Teach 48: 711–744, 2011