Application of the Test of Scientific Literacy Skills in the Assessment of a General Education Natural Science Program

Abstract

The peer-reviewed and psychometrically validated Test of Scientific Literacy Skills developed by Gormally et al. was used to assess the strengths and weaknesses of a general education natural science program. By comparing the scores of students who had already taken at least one course in this area with the scores of those who had not, and by relating scores to students' perceptions of learning about science, a richer understanding of our program has emerged. The ability to focus on particular skill sets and areas of proficiency has allowed us to identify specific areas to target for improvement.

Scientific Literacy Assessment Instruments: A Systematic Literature Review

“…Of the ten surveys that applied the TOSLS and its adapted versions, mean scores higher than 50% were found in the studies by Čipková et al (2020), Shaffer et al (2019), andWaldo (2014), and mean scores below this value were noted by Gomes and Almeida (2016), Kola et al (2020), Santiago et al (2020), and Utami and Hariastuti (2019). Gormally et al (2012) observed significant gains in science literacy skills on the post-test.…”

“…As for the TOSLS, Gormally et al (2012), Santiago et al (2020) and Souza (2019) reported the hit frequencies by skill, Shaffer et al (2019) presented them by skill categories and Cartwright et al (2020), Čipková et al (2020), Gomes and Almeida (2016), Kola et al ( 2020), Utami and Hariastuti (2019) and Waldo (2014) reported the hit frequencies of the instrument as a whole.…”

“…Among the studies that used the TOSLS, there was no positive or negative predominance of any of the skills. In the case of the best skills, the research by Santiago et al (2020) and Čipková et al (2020) identified the ability to evaluate the uses and misuses of scientific information; Souza (2019) found the ability to identify a valid scientific argument; Utami and Hariastuti (2019) checked the ability to solve problems using quantitative skills, including basic statistical analysis; Waldo (2014) identified category 1 skills, related to recognizing and analyzing the use of research methods that lead to scientific knowledge; and Gomes and Almeida (2016) looked at the abilities to identify a valid scientific argument, to evaluate the uses and misuses of scientific information, to understand the elements of a research design and how they impact scientific findings/conclusions, to create an appropriate graph from data, and to justify inferences, predictions, and conclusions based on quantitative data.…”

“…Of those who proceeded with the validation process, the studies by Atta et al ( 2020 Finally, of the studies that performed only the application of already developed and validated instruments, Camargo et al (2011), Sousa (2019a, 2019b), Gresczysczyn et al (2018), Lima and Garcia (2015), Lima and Garcia (2013), and Rivas et al (2017) used the TACB; Čipková et al (2020), Gomes and Almeida (2016), Shaffer et al (2019), Souza (2019), Utami andHariastuti (2019), andWaldo (2014) applied the TOSLS, Vizzotto (2019) and Vizzotto andDel Pino (2020a, 2020b) As for the level of education, 32 studies (74.4%) assessed the scientific literacy of students of only one level and 11 (25.6%) assessed individuals of different levels and even teachers. It can be noticed that there is a predominance of research conducted in order to assess the level of scientific literacy of secondary and higher education students, followed by those that assess students in the 3rd cycle of basic education, as shown in Figure 1.…”

Instrumentos De Avaliação Da Literacia Científica: Uma Revisão Sistemática De Literatura

“…Dentre os estudos que utilizaram o TOSLS, não houve predominância positiva ou negativa de nenhuma das habilidades. No caso das melhores habilidades, as pesquisas de Santiago et al (2020) e de Čipková et al (2020) identificaram a habilidade de avaliar os usos e os usos indevidos das informações científicas; Souza (2019) constatou a habilidade de identificar um argumento científico válido; Utami e Hariastuti (2019) verificaram a habilidade de resolver problemas usando habilidades quantitativas, incluindo análises estatísticas básicas; Waldo (2014) identificou as habilidades da categoria 1, relacionadas com o reconhecimento e com a análise do uso de métodos de investigação que levam ao conhecimento científico; e Gomes e Almeida (2016) observaram as habilidades de identificar um argumento científico válido, de avaliar os usos e os usos indevidos das informações científicas, de compreender os elementos de um projeto de pesquisa e como estes impactam nas descobertas/conclusões científicas, de criar um gráfico apropriado a partir dos dados e de justificar as inferências, previsões e conclusões com base em dados quantitativos.…”