Results of an Exploratory Analysis of PISA 2015 Survey of Student Participation in Outside-School-Time Programs

Abstract

The Program for International Student Assessment (PISA) by the OECD measures student study time during formal school periods and during periods of out-of-school-time (OST). The purpose of these items is to account for differences in country to country achievement levels. However, analyses of the impact of additional study time on student achievement have produced conflicting results across countries. While more time given to a school subject within formal school is positively related to achievement in that topic, more time spent on OST is negatively related to average achievement between and within countries. The paper proposes a reconceptualization of OST and achievement by integrating theoretical frameworks of study time, student abilities, and student feelings of efficacy. The results of a descriptive and conceptual analysis of a set of new survey items in the 2015 PISA for 22 countries shows that students benefit from additional study time by having increased feelings of efficacy in a school subject (such as science) but not in measurable levels of achievement. While country to country levels OST participation rates are different, the patterns of relationships between OST participation, student achievement, and attitudes are similar.

The contribution of research on out-of-school-time on educational theories and practice: A review of European research between 1999 and 2019

“…Bray examined case studies of individual countries, such as Ireland, England, Slovakia, Croatia, Poland, and Lithuania to illustrate how family background influenced motivation to attend OST. The patterns of differences between countries reported by Bray are consistent with PISA survey results in country rank order (OECD, 2012;Suter, 2019a, andspecial analysis of PISA 2018 unpublished).…”

“…Guill et al (2019), concluded that evidence from the longitudinal survey of German students showed that student grades in mathematics or language (German) were not affected by attending private tutoring nor by the instructional quality of the tutoring as they had expected. However, they found that student attitudes toward mathematics was higher for students after receiving tutoring lessons (Guill et al, 2019;see also;Suter, 2019a). This is a useful observation that should inspire other researchers to consider the non-academic as well as the academic influences of OST on student behavior.…”

“…On the other hand, the researchers noted that middle class parents who wished to give their children some advantages in competing for higher education would also able to use OST as a means of increasing those chances. The question of whether the relationship of tutoring to family status is biased against low status families is likely to remain an unsolved mystery because observational studies reviewed in this compilation generally conflict with evidence from large-scale research studies like PISA (Entrich, 2018;Suter, 2019a). The results of the first series of studies in Germany showed that participation in extra study programs was not related to family background, such as parent's educational level.…”

“…He noted that students of lower ability would be most likely to benefit from additional study time because they require more time and effort to reach achievement levels of higher ability students (Carroll, 1963(Carroll, , 1989. His theory has been cited by European researchers who developed frameworks for study time analysis in international studies Suter, 2016Suter, , 2019a) and it will guide the following analysis. Berliner (1990), in a delightful essay on the nature of time and learning, examined historical, philosophical, and recent research about how instructional time has been defined.…”

“…Even though a large number of studies have been conducted about the influence of additional study time on student achievement, agreement about the role of study time as an explanatory factor for student achievement and country differences in academic achievement has not been reached (Berliner, 1990;Farbman, 2015;Long, 2014;OECD, 2012;Rappleye & Komatsu, 2018;Suter, 2019a). In 1972, Torsten Hus en, then director of the IEA, aware of the increasing media interest in length of school time, raised concerns about appealing to study time as having a distinct role in country achievement levels (Hus en, 1972).…”