Issues in Survey Assessments of STS Courses

Abstract

This study presents the circuitous route taken in an attempt to identify an assessment mechanism for evaluating the impact of STS courses. After initially developing a survey that faculty felt only allowed for the measurement of student opinions about the impact of required STS general education courses, the faculty at Clemson University identified Aikenhead and Ryan's Views on Science-Technology-Society (VOSTS) as an instrument that, when used with the scoring guide created in this project, might be able to assess the sophistication of students views about science and technology in society. The results from administering the pre-and post-VOSTS surveys in one STS course revealed no significant differences when comparing the sophistication of students' views about STS before and after the course. These findings demonstrate that, among other considerations discussed in the manuscript, it will be difficult to find a way of assessing STS courses that can be used for the diverse courses meeting the requirement.C an we use a single survey to demonstrate what university Science, Technology and Society (STS) courses accomplish? Clemson University has had since 2005, a requirement that all students take a course dealing with Science and Technology in Society. As coordinator of this program, Dr. Pamela Mack has an interest both in demonstrating what such courses provide to students and in evaluating STS courses in different departments to make sure they address STS issues effectively. This paper discusses an effort to measure the impact of STS courses by using Aikenhead and Ryan's (1992) Views on Science-Technology-Society (VOSTS) instrument. We present our results here to further the discussion of how STS courses can be assessed. Development of the ProjectWhen the Clemson STS program became interested in evaluating STS courses taught in a wide range of departments, we first tried writing our own survey, but that proved very difficult. Graduate student Wes Ballard researched preexisting surveys, and we decided in fall 2006 that VOSTS would be the best instrument for evaluating STS courses. To assess STS courses in a variety of disciplines, we needed a survey that did not focus on specific knowledge but rather on thinking. We thought VOSTS addressed underlying concepts and analytical tools that should be common in all STS courses.We wanted a survey that would get beyond opinion and instead measure sophistication of thinking. In our initial effort to write our own survey, we asked student opinions about such statements as the following:• Based on the content of this course, I believe I developed new ways of thinking about issues relating to science, technology, and their relationship to society. • Citizens should be concerned about science and technology developments.Although we felt this provided us with information that could be helpful in continuously improving our courses, we were concerned that students might only tell us what they thought we wanted to hear. VOSTS

A Novel Approach to Teaching Undergraduates the Societal Impact of Robotics

“…This course meets the general education requirement at Clemson University in Science and Technology in Society (STS) [14], [15]:…”

Board # 45 : A Course on the Societal Impact of Robotics: Preliminary Outcomes

“…The original VOSTS survey contains over 100 multiple choice questions in seven categories [4]. The modified VOSTS survey [5] used here has 13 questions spanning 6 categories. The modified VOSTS survey is administered using Canvas.…”