Important conceptual and measurement issues with regard to research utilization could be better addressed if research in the area were undertaken longitudinally by multi-disciplinary teams of researchers.

We examined participants' reading and recall of informed consent documents presented via paper or computer. Within each presentation medium, we presented the document as a continuous or paginated document to simulate common computer and paper presentation formats. Participants took slightly longer to read paginated and computer informed consent documents and recalled slightly more information from the paginated documents. We concluded that obtaining informed consent online is not substantially different than obtaining it via paper presentation. We also provide suggestions for improving informed consent--in both face-to-face and online experiments.

Multiple choice (MC) questions from a graduate physiology course were evaluated by cognitive-psychology (but not physiology) experts, and analyzed statistically, in order to test the independence of content expertise and cognitive complexity ratings of MC items. Integration of higher order thinking into MC exams is important, but widely known to be challenging-perhaps especially when content experts must think like novices. Expertise in the domain (content) may actually impede the creation of higher-complexity items. Three cognitive psychology experts independently rated cognitive complexity for 252 multiple-choice physiology items using a six-level cognitive complexity matrix that was synthesized from the literature. Rasch modeling estimated item difficulties. The complexity ratings and difficulty estimates were then analyzed together to determine the relative contributions (and independence) of complexity and difficulty to the likelihood of correct answers on each item. Cognitive complexity was found to be statistically independent of difficulty estimates for 88 % of items. Using the complexity matrix, modifications were identified to increase some item complexities by one level, without affecting the item's difficulty. Cognitive complexity can effectively be rated by non-content experts. The six-level complexity matrix, if applied by faculty peer groups trained in cognitive complexity and without domain-specific expertise, could lead to improvements in the complexity targeted with item writing and revision. Targeting higher order thinking with MC questions can be achieved without changing item difficulties or other test characteristics, but this may be less likely if the content expert is left to assess items within their domain of expertise.

An MR is a tool, and the MR-EBM that we describe can be useful to develop or evaluate a curriculum in EBM. The MR tool is particularly compatible with the objectives of training for EBM and practice and can be applied to create or evaluate a curriculum using any topical KSA framework. The MR-EBM we describe could be adopted or adapted to represent other institutional objectives for EBM training.