A design attribute framework for course planning and learning assessment

Safoutin, Michael; Atman, Cynthia J.; Adams, Robin; Rutar, Teodora; Kramlich, John C.; Fridley, James L.

doi:10.1109/13.848072

Cited by 51 publications

(38 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…11) or project management (p=0.14) than their low IM counterparts. However, both groups (high and low IM) have significantly higher IM scores than the quasi-control cohorts as noted before.…”

Section: Assessment Methods and Analysismentioning

confidence: 97%

“…The test and quasi-control cohorts completed a number of assessment instruments, including the Situational Intrinsic Motivation Scale (SIMS) by Guay and Vallerand [10], and the design questionnaire (DQ) developed by Safoutin et al [11]. Because the SIMS has been shown to be both internal reliability externally valid, confirmatory factor analysis was used on its four scales: intrinsic motivation (IM), identified regulation, external regulation and amotivation.…”

Section: Assessment Methods and Analysismentioning

confidence: 99%

See 1 more Smart Citation

Converting Traditional Materials Labs to Project-based Learning Experiences: Aiding students' Development of Higher-order Cognitive Skills

Vanasupa¹,

Chen²,

Stolk

et al. 2007

MRS Proc.

View full text Add to dashboard Cite

Against a backdrop of compelling societal needs, graduates in science and engineering now must master their disciplines and demonstrate a sophisticated level of cognitive, affective and social development. This has lead a number of national and international commissions on science and engineering to urge educators to re-think the way in which STEM disciplines are taught. We have chosen to "repackage" a traditional undergraduate materials engineering curriculum in a form designed to promote the development of higher-order cognitive skills like self-directed learning and design. Classic metallurgy experiments have been converted to project-based learning experiences where students are put in the role of "designers" of problem solutions and faculty play the role of coaches. These include: designing, prototyping and marketing of a cast metal object; systems designing, building and testing of a fiber optic spectrometer; product improvement of a prosthetic device; evaluation of oxidation process for production; design and evaluation of a heat treatment process for roller bearings; and materials characterization of an everyday product. Projects were designed to leverage known relationships within the educational psychology literature that enable deeper learning. Evaluation of 36 juniors in a project-based learning course (i.e., the test cohort) against a quasi-control group in traditional engineering courses showed that the test cohort scored significantly higher on two motivation scales shown to be critical components in self-directed learning (p<0.001). The test cohort also reported a significantly higher use of peers as learning resources than the quasicontrol group. Their motivation scores also correlate highly with self-reported comfort with several aspects of design, implying that their motivation contributes significantly to students' ability to effectively engage in the design process. In this paper, we present examples of the materials engineering projects that were designed and implemented, and the design features that enable them to promote the development of sophisticated cognitive functioning.

show abstract

Section: Assessment Methods and Analysismentioning

confidence: 97%

Section: Assessment Methods and Analysismentioning

confidence: 99%

Converting Traditional Materials Labs to Project-based Learning Experiences: Aiding students' Development of Higher-order Cognitive Skills

Vanasupa¹,

Chen²,

Stolk

et al. 2007

MRS Proc.

View full text Add to dashboard Cite

show abstract

“…Safoutin et al (2000) identified 13 components of the ABET criterion 3c: an ability to design a system, component, or process to meet desired needs within realistic constraints (ABET, 2011). The components were: need recognition, problem definition, planning, information gathering, idea generation, modelling, evaluation, feasibility analysis, selection, implementation, documentation, communication, and iteration (Safoutin et al, 2000). There is a substantial overlap with the components of problem solving identified by Besterfield-Sacre et al (2000), discussed earlier.…”

Section: Qualitative Models and Communicationmentioning

confidence: 99%

“…There is a substantial overlap with the components of problem solving identified by Besterfield-Sacre et al (2000), discussed earlier. Safoutin et al (2000) designed a freshman design course for engineering students based on the identified components of designing ability. After the course both the students and the instructor completed a survey addressing the students' performance on the 13 components.…”

Section: Qualitative Models and Communicationmentioning

confidence: 99%

Does teaching students how to explicitly model the causal structure of systems improve their understanding of these systems?

Jensen

2014

European Journal of Engineering Education

View full text Add to dashboard Cite

Does teaching students how to explicitly model the causal structure of systems improve their understanding of these systems?. European Journal of Engineering Education, 39(4): 391-411Access to the published version may require subscription. N.B. When citing this work, cite the original published paper. Permanent link to this version:http://urn.kb.se/resolve?urn=urn:nbn:se:fhs:diva-5242 1 Does teaching students how to explicitly model the causal structure of systems improve their understanding of these systems?If students really understand the systems they study, they would be able to tell how changes in the system would affect a result. This demands that the students understand the mechanisms that drive its behaviour. The study investigates potential merits of learning how to explicitly model the causal structure of systems. The approach and performance of 15 system dynamics students, who are taught to explicitly model the causal structure of the systems they study, was compared with the approach and performance of 22 engineering students, who do generally not receive such training. The task was to bring a computer simulated predator-and-prey ecology to equilibrium. The system dynamics students were significantly more likely than the engineering students to correctly frame the problem. They were not much better at solving the task, however. It seemed that they had only learnt how to make models and not how to use them for reasoning.Keywords: engineering education research; modelling; external representations, dynamic systems, qualitative reasoning IntroductionThe literature on modelling tends to focus on the quantitative model that results when the laws of physics have been applied and relevant measurements made. The goal of the modelling process is a model that lends itself to mathematical treatment (e.g., Lesh & Doerr, 2003;Redish & Smith, 2008). The underlying qualitative model appears to be seen as self-evident and/or implicit in the quantitative model. True as this may be,we cannot be certain that it is evident to the students. When the mathematical model has been manipulated and transformed, and a result calculated, the result has to be given meaning (Bissel & Dillon, 2000; Kehler & Lester, 2003;Redish & Smith, 2008). The students need to understand why a result is as it is. In other words, they have to be able to interpret their results. Furthermore, if they 2 really understand the system under study they would be able to tell changes in the system would affect a result. This demands that the students understand the qualitative structure of the system, i.e., the mechanisms that drive its behaviour.The aim of this study is to investigate the potential merits of training students in how to explicitly model the qualitative, or causal, structure of the systems they study.Do students who receive such training understand the systems they study better than students who do not? It might be the case that students spontaneously model the causal structure when the need arises, or that it is sufficient that the cau...

show abstract

“…Based on the literature and existing collections of learning outcomes (e.g. 7,8,9,10,11,12 ), we created an extensive hierarchical list of learning outcomes and their more specific, measurable indicators. This list of outcomes and indicators was validated through expert consultation with engineering design instructors at the university.…”

Section: Draft Rubric Developmentmentioning

confidence: 99%

User Testing with Assessors to Develop Universal Rubric Rows for Assessing Engineering Design

Dawe

Romkey

McCahan

et al.

2016 ASEE Annual Conference &Amp; Exposition Proceedings

View full text Add to dashboard Cite

This paper describes the process of testing and refining modular rubric rows developed for the assessment of engineering design activities. This is one component of a larger project to develop universal analytic rubrics for valid and reliable competency assessment across different academic disciplines and years of study. The project is being undertaken by researchers based in the Faculty of Applied Science and Engineering at the University of Toronto.From January 2014 to June 2015, we defined and validated indicators (criteria) for engineering design, communication, and teamwork learning outcomes, then created descriptors for each indicator to discriminate between four levels of performance: Fails, Below, Meets, and Exceeds graduate expectations. From this rubric bank, applicable rows can be selected and compiled to produce a rubric tailored to a particular assessment activity. Here we discuss these rubrics within the larger context of learning outcomes assessment tools for engineering design.We tested draft rubrics in focus group sessions with assessors (teaching assistants and course instructors who assess student work in engineering design). We followed the testing with structured discussions to elicit feedback on the quality and usability of these rubrics, and to investigate how the assessors interpreted the language used in the indicators and descriptors. We asked participants to identify indicators they believed were irrelevant, redundant, or missing from the rubric. We also asked them to identify and discuss indicators and descriptors that were confusing. Finally, we asked them what changes they would recommend and what training materials they would find useful when using rubrics of this design.We analysed the consistency of assessor ratings to identify problematic rows and used qualitative feedback from follow-up discussions to better understand the issues that assessors had with the rubrics. Three of the six engineering design rubric items we tested showed evidence of rater inconsistency, uncertainty, and indecision. While some rubric rows received similar criticism from most participants, we also identified differences in assessors' rubric design preferences and in how they apply rubrics to evaluate student work. It also emerged that assessors have different conceptions of engineering design and the design process, and are confused when presented with unfamiliar terminology. Based on our interpretations, we identified changes we should make to improve the rubric structure and content.Our aim is to inform other educators who may be developing tools for the assessment of engineering design by sharing our methodology and discussing the feedback received. Engineering educators could adopt or adapt our user testing methodology to improve the usability of similar assessment tools. Our discoveries about rubric structure improvements could be explored further to define best practices in the design of universal rubrics. Our next steps include applying what we have learned to refine the rubrics and devel...

show abstract

A design attribute framework for course planning and learning assessment

Cited by 51 publications

References 8 publications

Converting Traditional Materials Labs to Project-based Learning Experiences: Aiding students' Development of Higher-order Cognitive Skills

Converting Traditional Materials Labs to Project-based Learning Experiences: Aiding students' Development of Higher-order Cognitive Skills

Does teaching students how to explicitly model the causal structure of systems improve their understanding of these systems?

User Testing with Assessors to Develop Universal Rubric Rows for Assessing Engineering Design

Contact Info

Product

Resources

About