An analysis of guidelines for expert reviewers

Abstract

Forty eight sets of recommendations identified by their authors as practical heuristics for the evaluation and revision of instructional materials were reviewed and consolidated. These guidelines were extracted from professional journals, book chapters and independent publications. Initially, all items were compiled, and then sorted into three categories according to their specific focus: content (subject matter), design, and presentation. In a second sort, identical items were eliminated and semantically equivalent items were grouped together. Three independent judges performed the same tasks for reliability. The outcome is a comprehensive list of 67 items, representing all of the reviewed guidelines. This instrument could be a successful aid in identifying deficiencies of instructional materials in the areas of content, design and presentation.

Enabling meaningful learning through Web-based instruction with occupational therapy students

“…These integrated instructional principles, comprising five categories, address all seven attributes of meaningful learning. The first four categories, drawn from Saroyan and Geis (1988) are: (1) instructional (course) design, or pedagogical issues and strategies; (2) subject matter, or content issues including the TA and MOHO frameworks; (3) language or semantic and syntactic issues such as clarity, vocabulary and consistency of terminology within the instructions of each analysis task; and (4) presentation such as graphics, format and layout. Weston et al (1999) identified a fifth category of issues to extend the framework to the on-line context, such as student access, student computer literacy, infrastructure, navigation, loading speed and bandwidth.…”

Environment Analysis: How Do You Catch the Learner?

“…Where are the support resources (library, laboratory, tutors) located in relation to the classroom sub-system? Many an independent study course has failed because the independent study materials are located in a library or learning resources unit that is inaccessible to teachers, peers, or tutors who can help the student when help is needed (Saroyan and Geis, 1988). Understanding this factor, designers can recommend relocation of the materials or change their content to provide more self-analysis and feedback.…”

“…The instructional situation must be analysed, to determine if the roles of the learners and teachers can be fulfilled in the environment (Saroyan and Geis, 1988). At the sub-system level of the classroom, the physical environment of the room is crucial.…”

Differences in expert practice: a case from formative evaluation

“…(The same classification was used in a previous study by Saroyan and Geis [1988] in which three independent judges sorted comments with 97% reliability.) The most salient distinction between the two groups was observed in the scope of comments ( Table 2).…”