Criterion-related validities and inter-rater reliabilities for structured employment interview studies using situational questions (e.g. ''Assume that you were faced with the following situation . . . what would you do?'') were compared meta-analytically with studies using past behaviour questions (e.g. ''Can you think of a time when . . . what did you do?''). Validities and reliabilities were further analysed in terms of whether descriptively-anchored rating scales were used to judge interviewees' answers, and validities for each question type were also assessed across three levels of job complexity.While both question formats yielded high validity estimates, studies using past behaviour questions, when used with discriptively anchored answer rating scales, yielded a substantially higher mean validity estimate than studies using the situational question format with descriptively-anchored answer rating scales (.63 versus .47). Question type (situational versus past behaviour) was found to moderate interview validity, after controlling for whether studies used answer rating scales. No support was found for the hypothesis that situational questions are less valid for predicting job performance in high-complexity jobs.Sample-weighted mean inter-rater reliabilities were similar for both situational and past behaviour questions, provided that descriptively-anchored rating scales were used (.79 and .77, respectively), although they were slightly lower (.73) for past behaviour question studies lacking such rating scales.
A recently implemented research and development (R&D) programme in New Zealand is attempting to implement co-innovation principles throughout the country's agricultural sector. It is based on an agricultural innovation systems (AIS) approach, using five innovation platforms (IPs) based in the leading industries, plus a national-level IP. This paper presents and analyses the emerging challenges of operationalizing transdisciplinary research connected to co-innovation in the context of the programme. The main challenges relate to managing the complexity of a multi-stakeholder network, aligning the formal procedures for research funding and IPs with a co-innovation approach that requires flexibility, and changing participants' framing of innovation from linear to interactive models. Our conclusion is that 'learning by doing' is essential in operationalizing coinnovation. Its practical implications still need to be translated into institutional changes in the national R&D structures so that policies, instruments and incentives enable co-innovation. It is envisaged that the higher-level innovation platform will drive these changes.
Reflexive monitors (RMs) have been found to be vital to the success of co-innovation projects. While the practices utilized by RMs have been examined in various contexts, we examine the roles they have played in a new cultural context in New Zealand (NZ) and how it has been possible to embed these roles in a diverse range of innovation projects in the primary sector. This article will address this gap in terms of explaining the case-specific behaviours that have been utilized in six different co-innovation projects in the NZ agricultural innovation system. Qualitative data from interviews with five RMs will be used to argue that RMs are a key component in the co-innovation process and are required to play diverse roles depending on project circumstances to enhance system innovation-for example, devil's advocate, project supporter, consensus seeker, conflict mediator, critical enquirer or encourager. The findings have implications for the characteristics that make a good RM in terms of openness to new ideas, facilitation and critical thinking skills and how they report on the practice of monitoring a project reflexively utilizing monitoring and evaluation techniques.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.