Assessment of teaching effectiveness: Lack of alignment between instructors, institutions, and research recommendations

Self Cite

104

[This paper is part of the Focused Collection on Preparing and Supporting University Physics Educators.] The lack of knowledge about how to effectively spread and sustain the use of researchbased instructional strategies is currently a significant barrier to the improvement of undergraduate physics education. In this paper we address this lack of knowledge by reporting on an interview study of 35 physics faculty, of varying institution types, who were self-reported users of, former users of, or knowledgeable nonusers of the research-based instructional strategy Peer Instruction. Interview questions included in this analysis focused on the faculty's experiences, knowledge, and use of Peer Instruction, along with general questions about current and past teaching methods used by the interviewee. The primary findings include the following: (i) Faculty self-reported user status is an unreliable measure of their actual practice. (ii) Faculty generally modify specific instructional strategies and may modify out essential components. (iii) Faculty are often unaware of the essential features of an instructional strategy they claim to know about or use. (iv) Informal social interactions provide a significant communication channel in the dissemination process, in contrast to the formal avenues of workshops, papers, websites, etc., often promoted by change agents, and (v) experience with researchbased strategies as a graduate student or through curriculum development work may be highly impactful. These findings indicate that educational transformation can be better facilitated by improving communication with faculty, supporting effective modification by faculty during implementation, and acknowledging and understanding the large impact of informal social interactions as a mode of dissemination.

Section: Research Design and Methodologymentioning

confidence: 99%

How faculty learn about and implement research-based instructional strategies: The case of Peer Instruction

Dancy

Henderson

Turpen

2016

Self Cite

104

“…Physics education researchers have produced a plethora of research-based assessment (RBA) tools [1], but their use by faculty is often limited [2], in part because faculty members do not perceive these materials as meeting their genuine needs. Faculty work within diverse environments, constrained by diverse structures that may not match the environments and structures of materials' developers.…”

Section: Introductionmentioning

confidence: 99%

Research-based assessment affordances and constraints: Perceptions of physics faculty

Madsen

McKagan

Martinuk³

et al. 2016

[This paper is part of the Focused Collection on Preparing and Supporting University Physics Educators.] To help faculty use research-based materials in a more significant way, we learn about their perceived needs and desires and use this information to suggest ways for the physics education research community to address these needs. When research-based resources are well aligned with the perceived needs of faculty, faculty members will more readily take them up. We used phenomenographic interviews of ordinary physics faculty and department chairs to identify four families of issues that faculty have around research-based assessments (RBAs). First, many faculty are interested in using RBAs but need help with the practicalities of administering RBAs: how to find them, which ones there are, and how to administer them. Second, at the same time, many faculty think that RBAs are limited and do not measure many of the things they care about, or are not applicable in their classes. They want assessments to measure skills, perceptions, and specific concepts. Third, many faculty want to turn to communities of other faculty and experts to help them interpret their assessment results and suggest other ways to do assessment. They want to better understand their assessment results by comparing to others and interacting with faculty from other schools to learn about how they do assessment. Fourth, many faculty consider their courses in the broader contexts of accountability and their departments. They want help with assessment in these broader contexts. We also discuss how a faculty member's role in their department and type of institution influence their perceived wants and needs around assessment.

“…Working towards an alternative goal, introducing faculty participants to education research results that are new to them might improve their ability to try innovative strategies in environments where institutional or departmental pressures constrain their teaching. In particular, physics faculty could justify their decision to use active learning strategies to resistant administrators using education research findings and methods, e.g., using concept inventories to measure student outcomes [86][87][88]. That said, we caution that contrary to popular belief, quantitative research results often do not contribute to convincing individual faculty to initially try RBIS [19,87], and we argue that a workshop that focuses on this exclusively is unlikely to shift faculty participants' willingness to try new teaching strategies.…”

Section: Student Experiencesmentioning

confidence: 99%

Assessing the interactivity and prescriptiveness of faculty professional development workshops: The real-time professional development observation tool

Olmstead

Turpen

2016

Self Cite

Professional development workshops are one of the primary mechanisms used to help faculty improve their teaching, and draw in many STEM instructors every year. Although workshops serve a critical role in changing instructional practices within our community, we rarely assess workshops through careful consideration of how they engage faculty. Initial evidence suggests that workshop leaders often overlook central tenets of education research that are well established in classroom contexts, such as the role of interactivity in enabling student learning [S. Freeman et al., Proc. Natl. Acad. Sci. U.S. A. 111, 8410 (2014)]. As such, there is a need to develop more robust, evidence-based models of how best to support faculty learning in professional development contexts, and to actively support workshop leaders in relating their design decisions to familiar ideas from other educational contexts. In response to these needs, we have developed an observation tool, the real-time professional development observation tool (R-PDOT), to document the form and focus of faculty engagement during workshops. In this paper, we describe the motivation and methodological considerations behind the development of the R-PDOT, justify our decisions to highlight particular aspects of workshop sessions, and demonstrate how the R-PDOT can be used to analyze three sessions from the Physics and Astronomy New Faculty Workshop. We also justify the accessibility and potential utility of the R-PDOT output as a reflective tool using preliminary data from interviews with workshop leaders, and consider the roles the R-PDOT could play in supporting future research on faculty professional development.