Effects of a change to more formative assessment among tertiary mathematics students

Abstract

A change in teaching delivery at a large Australian university, from two semesters to three trimesters, was the occasion for using more formative assessment in a core first-year mathematics unit. This study compared evidence about learning outcomes for two cohorts in adjacent years. Cohort 1 was the last taught over a semester, and Cohort 2 the first taught over a trimester. There was no change in overall workload, and no change in the unit's total teaching hours, syllabus or materials. Assessments were changed for class tests during the teaching period by giving Cohort 2 access to unlimited practice and computer-assisted feedback on the questions in the test database, followed by doing the tests under examination conditions. For Cohort 2, a written assignment was also added, focused on giving a clear solution to a mathematics problem, and awareness of the need for appropriate evidence, both background and internal to the problem. Learning outcomes were compared using closely comparable tasks from the final examinations, and examining students' answers in the examination scripts. Outcomes were assessed by a method derived from the solo taxonomy, which afforded a common scale to measure the quality of learning outcomes observable in final examination scripts. Results on separate tasks, plus those for a composite score, favoured Cohort 2. The effect size for the composite score was 0.457. This indicates that the unlimited practice with computer feedback for class tests, and the writing assignment, were functioning as intended in promoting learning with understanding. References S. Bengmark, H. Thunberg, and T. M. Winberg. Success-factors in transition to university mathematics. Int. J. Math. Ed. Sci. Tech., 48(7):988–1001, 2017. doi:10.1080/0020739X.2017.1310311. J. B. Biggs and K. F. Collis. Evaluating the quality of learning: The SOLO taxonomy. Academic Press, New York, 1981. URL https://www.elsevier.com/books/evaluating-the-quality-of-learning/biggs/978-0-12-097552-5. A. Bowen-James. Perceptions of learning environments among tertiary mathematics students. Sc.Ed.D. Thesis. Curtin University of Technology, 2002. H. Chick, J. M. Watson, and K. F. Collis. Using the solo taxonomy for error analysis in mathematics. Res. Math. Ed. Aust., 1(1):34–47, 1988. M. R. Freislich. A comparison between the effects of Keller Plan instruction and traditional teaching methods on the structure of learning outcomes among tertiary mathematics students. Sc.Ed.D. Thesis. Curtin University of Technology, 1997. M. R. Freislich. The effects of Keller Plan instruction on the achievement and attitudes of tertiary mathematics students. Proc. Int. Conf. Teach. Math., Istanbul. 2006. M. Gill and M. Greenow. How effective is feedback in computer-aided assessment? Learn. Media Tech., 33(3):207–220, 2008. doi:10.1080/17439880802324145. J. Hannah, A. James, and P. Williams. Does computer-aided formative assessment improve learning outcomes? Int. J. Math. Ed. Sci. Tech., 45(2):269–281, 2014. doi:10.1080/0020739X.2013.822583. D. Harris and M. Pampaka. \T1\textquoteleft they [the lecturers] have to get through a certain amount in an hour\T1\textquoteright : first year students\T1\textquoteright problems with service mathematics lectures. Teach. Math. App., 35(3):144–158, 2016. doi:10.1093/teamat/hrw013. S. Higgins and M. Katsipataki. Communicating comparative findings from meta-analysis in educational research: some examples and suggestions. Int. J. Math.. Res. Meth. Ed., 39(3):237–254, 2016. doi:10.1080/1743727X.2016.1166486. P. W. Hillock and R. N. Khan. A support learning programme for first-year mathematics. Int. J. Math. Ed. Sci. Tech., 50(7):24–29, 2019. doi:10.1080/0020739X.2019.1656830. A. Hodge, J. C. Richardson, and C. S. York. The impact of a web-based homework tool in university algebra courses on student learning and strategies. J. Online Learn. Teach., 5(4):618–629, 2009. URL https://jolt.merlot.org/vol5no4/hodge_1209.htm. D. Holton and D. Clarke. Scaffolding and metacognition. Int. J. Math. Ed. Sci. Tech., 37(2):127–143, 2006. doi:10.1080/00207390500285818. A. H. Jonsdottir, A. Bjornsdottir, and G. Stefansson. Difference in learning among students doing pen-and-paper homework compared to web-based homework in an introductory statistics course. J. Stat. Ed., 25(1):12–20, 2017. doi:10.1080/10691898.2017.1291289. M. McAlinden and A. Noyes. Mathematics in the disciplines at the transition to university. Teach. Math. App., 38(2):61–73, 2019. doi:10.1093/teamat/hry004. J. Nicholas, L. Poladian, J. Mack, and R. Wilson. Mathematics preparation for university: entry pathways and their effect on performance in first year mathematics and science subjects. Int. J. Innov. Sci. Math. Ed., 23(1):37–51, 2015. https://openjournals.library.sydney.edu.au/index.php/CAL/article/view/8488. M. I. Nunez-Pena, R. Bono, and M. Suarez-Pellicioni. Feedback on students' performance: a possible way of reducing the negative effect of math anxiety in higher education. Int. J. Ed. Res., 70(1):80–87, 2015. doi:10.1016/j.ijer.2015.02.005. J. T. E. Richardson. Student learning in higher education: a commentary. Ed. Psych. Rev., 29(1):353–362, 2017. doi:10.1007/s10648-017-9410-x. L. J. Rylands and D. Shearman. Mathematics learning support and engagement in first year engineering. Int. J. Math. Ed. Sci. Tech., 49(8):1133–1147, 2018. doi:10.1080/0020739X.2018.1447699. K. A. Seaton. Efficacy and efficiency in formative assessment: an informed reflection on the value of partial marking. Int. J. Math. Ed. Sci. Tech., 44(7):963–971, 2013. doi:10.1080/0020739X.2013.831490. D. Wood, J. S. Bruner, and G. Ross. The role of tutoring in problem solving. J. Child Psychol. Psych., 17(1):89–100, 1976. doi:10.1111/j.1469-7610.1976.tb00381.x. L. Zetterqvist. Applied problems and use of technology in an aligned way in basic courses in probability and statistics for engineering students—a way to enhance understanding and increase motivation. Teach. Math. App., 36(2):108–122, 2017. doi:10.1093/teamat/hrx004.

Online Support for Tertiary Mathematics Students in a Blended Learning Environment

“…The SOLO taxonomy has been used at tertiary level as a framework for defining intended learning outcomes for programs in mathematics and computer science [24] and its application in other science disciplines at tertiary level has been found to be a valuable diagnostic tool [25]. The SOLO levels were adapted for the work reported in [26] to define a method of evaluating levels of learning outcomes in tertiary students' mathematics.…”