Despite growing interest in public engagement with research, there are many challenges to evaluating engagement. Evaluation findings are rarely shared or lead to demonstrable improvements in engagement practice. This has led to calls for a common 'evaluation standard' to provide tools and guidance for evaluating public engagement and driving good practice. This paper proposes just such a standard. A conceptual framework summarizes the three main ways in which evaluation can provide judgements about, and enhance the effectiveness of, public engagement with research. A methodological framework is then proposed to operationalize the conceptual framework. The standard is developed via a literature review, semi-structured interviews at Queen Mary University of London and an online survey. It is tested and refined in situ in a large public engagement event and applied post hoc to a range of public engagement impact case studies from the Research Excellence Framework. The goal is to standardize good practice in the evaluation of public engagement, rather than to use standard evaluation methods and indicators, given concerns from interviewees and the literature about the validity of using standard methods or indicators to cover such a wide range of engagement methods, designs, purposes and contexts. Adoption of the proposed standard by funders of public engagement activities could promote more widespread, high-quality evaluation, and facilitate longitudinal studies to draw our lessons for the funding and practice of public engagement across the higher education sector.
Background Measurements of residual limb volume often guide decisions on the type and timing of prosthetic prescription. To help inform these decisions, it is important that clinicians use measurement tools that are reliable and valid. Purpose The aim of this systematic review was to investigate the reliability and validity of measurement tools for residual limb volume in people with limb amputations. Data Sources A comprehensive search on MEDLINE, EMBASE, CINAHL, Scopus, and Web of Science was performed on July 11, 2016. Study Selection Studies were included if they examined the reliability or validity of measurement tools for residual limb volume, were conducted on humans, and were published in English. Data Extraction Data were extracted from 11 reliability and 4 validity studies and included study characteristics, volumetric estimates, and reliability and validity estimates. The quality of the studies was also rated. Data Synthesis Data from 2 studies (38 participants) indicated good to excellent intrarater (intraclass correlation coefficient [ICC] ≥0.88) and interrater (ICC ≥0.88) reliability and high between-session reliability (coefficient of variation [CV] = 10%) for water displacement volumetry. One study (28 participants) reported excellent intrarater and interrater reliability (ICC ≥0.93) for the circumferential method, and data from 2 studies (19 participants) indicated high between-session reliability for the optical surface scanner (CV ≤9.8%). Three studies (26 participants) indicated good to excellent between-session reliability results for computed tomography (CV = 9.2%–10.9%). One study (7 participants) showed moderate within-session reliability (CV = 50%). Using water displacement volumetry as the gold standard, 2 studies (79 participants) indicated excellent validity for the circumferential method ( r ≥0.92; ICC ≥0.92). All studies reporting measures of reliability or validity were performed with people who had transtibial amputations. Limitations Only studies published in English and in which water displacement volumetry was used as the gold standard were included in this review. The reliability and validity of the quality rating scale used in this review have not been tested. Conclusions On the basis of a limited number of moderate- to high-quality studies with small sample sizes, circumferential and water displacement methods were found to be reliable, and the circumferential method was found to be valid in people with transtibial amputations. There are inadequate data for drawing conclusions about volume measurement methods in people with other types of limb amputations.
Background: Residual limb volume is often measured as part of routine care for people with amputations. These measurements assist in the timing of prosthetic fitting or replacement. In order to make well informed decisions, clinicians need access to measurement tools that are valid and reliable. Objectives: To assess the reliability and criterion validity of the iSense optical scanner in measuring volume of transtibial residual limb models. Study Design: Three assessors performed two measurements each on 13 residual limb models with an iSense optical scanner (3D systems, USA). Intra-rater and inter-rater reliability were calculated using intraclass correlation coefficients. Bland Altman plots were inspected for agreement. Criterion validity was assessed using a steel rod of known dimensions. Ten repeated measurements were performed by one assessor. A t-test was used to determine differences between measured and true rod volume. Results: Intra-rater reliability was excellent (range of intraclass correlation coefficients: 0.991–0.997, all with narrow 95% confidence intervals). While the intraclass correlation coefficients suggest excellent inter-rater reliability between all three assessors (range of intraclass correlation coefficients: 0.952–0.986), the 95% confidence intervals were wide between assessor 3 and the other two assessors. Poor agreement with assessor 3 was also seen in the Bland-Altman plots. Criterion validity was very poor with a significant difference between the mean iSense measurement and the true rod volume (difference: 221.18 mL; p < 0.001). Conclusions: Although intra-rater reliability was excellent for the iSense scanner, we did not find similar results for inter-rater reliability and validity. These results suggest that further testing of the iSense scanner is required prior to use in clinical practice. Clinical relevance The iSense offers a low cost scanning option for residual limb volume measurement. Intra-rater reliability was excellent, but inter-rater reliability and validity were such that clinical adoption is not indicated at present.
Background: The clinical utility of measuring pressure at the prosthetic socket-residual limb interface is currently unknown. Objectives: This study aimed to identify whether measuring interface pressure during prosthetic design and fabrication results in closer agreement in pressure measurements between sockets made by different clinicians, and a reduction in pressure over areas of concern. It also investigated whether clinicians value knowing the interface pressure during the fabrication process. Study design: Mixed methods. Methods: Three prosthetists designed a complete prosthetic system for a transtibial residual limb surrogate. Standardised mechanical testing was performed on each prosthetic system to gain pressure measurements at four key anatomical locations. These measurements were provided to the clinicians, who subsequently modified their sockets as each saw fit. The pressure at each location was re-measured. Each prosthetist completed a survey that evaluated the usefulness of knowing interface pressures during the fabrication process. Results: Feedback and subsequent socket modifications saw a reduction in the pressure measurements at three of the four anatomical locations. Furthermore, the pressure measurements between prosthetists converged. All three prosthetists found value in the pressure measurement system and felt they would use it clinically. Conclusions: Results suggest that sensors measuring pressure at the socket-limb interface has clinical utility in the context of informing prosthetic socket design and fabrication. If the technology is used at the check socket stage, iterative designs with repeated measurements can result in increased consistency between clinicians for the same residual limb, and reductions in the magnitudes of pressures over specific anatomical landmarks. Clinical relevance This study provides new information on the value of pressure feedback to the prosthetic socket design process. It shows that with feedback, socket modifications can result in reduced limb pressures, and more consistent pressure distributions between prosthetists. It also justifies the use of pressure feedback in informing clinical decisions.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.