Development and Testing of Program Evaluation Instruments for the iCook 4-H Curriculum

Abstract

Objective: To develop and test the validity of program outcome evaluation instruments for cooking, eating, and playing together for obesity prevention during iCook 4-H. Design: Instrument development for both youth and adults through pre-post testing of items newly constructed and compiled to address key curriculum constructs. Testing occurred throughout program intervention and dissemination to determine dimensionality, internal consistency and test-retest reliability, and validity. Setting: A 5-state out-of-school program in cooperative extension and other community sites. Participants: Youths aged 9−10 years; adults were main food preparers; the first phase involved 214 dyads and the second phase, 74 dyads. Main Outcome Measure(s): Youth measures were cooking skills, culinary self-efficacy, physical activity, and openness to new foods. Adult measures were cooking together, physical activity, and eating together. Analysis: Exploratory factor analysis to determine initial scale structure and confirmatory factor analysis to confirm factor structures. Longitudinal invariance tests to see whether the factor structure held over time. Test-retest reliability was determined by Pearson r and internal consistency was determined by coefficient V and Cronbach a. Validity testing was determined by Pearson r correlations. Results: Youth cooking skills, openness to new foods, and adult eating together and cooking together showed strong evidence for dimensionality, reliability, and validity. Youth physical activity and adult physical activity measures showed strong evidence for dimensionality and validity but not reliability. The youth culinary selfefficacy measure showed strong evidence for reliability and validity but weaker evidence for dimensionality. Conclusions and Implications: Program outcome evaluation instruments for youths and adults were developed and tested to accompany the iCook 4-H curriculum. Program leaders, stakeholders, and administrators may monitor outcomes within and across programs and generate consistent reporting.

Development and Validation of a New Home Cooking Frequency Questionnaire: A Pilot Study

“…Finally, the reliability tests showed a wide range of Î coefficients or ICC coefficients, depending on the nature of the variables. These results are in line with other studies aimed at validating food and/or culinary skill questionnaires, although these studies analyzed the reliability of their tools using Spearman or Pearson correlation tests instead of the ICC test [18,19,22,24]. Overall, our results showed that the HCFQ has the ability to produce a similar score with the same participants under the same conditions over time and is therefore suitable for studying the effectiveness of cooking training interventions.…”

Incorporating Technology Into the iCook 4-H Program, a Cooking Intervention for Adults and Children: Randomized Controlled Trial

“…Assessments at these time points included measuring children’s height, weight, waist circumference, and blood pressure as well as completing surveys. Survey questions assessed demographics, dietary intake, food security, cooking frequency, and cooking self-efficacy; program evaluation questions focused on cooking skills, family meals, physical activity, and goal setting [23-31]. We added questions after 12 months of the project to assess engagement with technological self-efficacy (ie, accessing the internet, and creating and uploading digital photos and videos to a study website).…”

Development of the iCook 4-H Curriculum for Youth and Adults: Cooking, Eating, and Playing Together for Childhood Obesity Prevention

“…Program leaders and assistants provided feedback to researchers about the curriculum through quick online process evaluations (Qualtrics, Qualtrics Labs Inc., Provo, UT; 2013) at the end of each session during the 4 phases of testing. 46,47 During the intervention pilot test and study, leaders were provided with participant process evaluation feedback so that they could make adjustments as needed; during the dissemination pilot test and study, leaders were provided with feedback upon request. The curriculum development team used leader feedback to address issues and questions.…”

“…Throughout each study phase, the fidelity of curriculum implementation was assessed. Fidelity of implementation evaluations 46,47 were conducted to determine whether sessions were implemented as intended. After a model fidelity tool was investigated during the pilot test, sessionspecific tools were developed to gather attendance and the timing of sections, and to assess the engagement of participants and leader effectiveness.…”