Abstract. Impact evaluation in public engagement necessarily requires measuring change. However, this is extremely challenging for drop-in activities due to their very nature. We present a novel method of impact evaluation which integrates graffiti walls into the experience both before and after the main drop-in activity. The activity in question was a soundscape exhibit, where young families experienced the usually inaudible sounds of near-Earth space in an immersive and accessible way. We apply two analysis techniques to the captured before and after data – quantitative linguistics and thematic analysis. These analyses reveal significant changes in participants' responses after the activity compared to before, namely an increased diversity in language used to describe space and altered conceptions of what space is like. The results demonstrate that the soundscape was surprisingly effective at innately communicating key aspects of the underlying science simply through the act of listening. The impacts also highlight the power of sonification in stimulating public engagement, which, through reflection, can lead to altered associations, perceptions, and understanding. Therefore, we show that this novel approach to drop-in activity evaluation, using graffiti walls both before and after the activity and applying rigorous analysis to this data, has the power to capture change and, thus, have a short-term impact. We suggest that commonly used evaluation tools suitable for drop-in activities, such as graffiti walls, should be integrated both before and after the main activity in general, rather than only using them afterwards as is typically the case.
Abstract. Impact evaluation in public engagement necessarily requires measuring change, however this is extremely challenging for drop-in activities due to their very nature. We present a soundscape exhibit, where young families experienced the usually inaudible sounds of near-Earth space, which used a novel method of evaluation integrating pre- and post- graffiti walls into the activity. We apply two analysis techniques to the captured before and after data: 1) Quantitative linguistics – Applying Zipf's law (the power law statistics of words) reveals an increased diversity of language concerning space afterwards, highlighting participants engaged with and reflected upon the sounds; 2) Thematic analysis – Finding and grouping patterns in the qualitative data shows altered conceptions of space around aspects of sound, dynamism, emptiness and electricity, areas highly relevant to the underlying space plasma physics of the sonified data. Therefore, we demonstrate that this novel approach to drop-in activity evaluation has the power to capture change from before to after, and thus short-term impact – specifically in this case showing the power of data sonification in innately communicating science. We suggest the method could be adopted by others in their drop-in engagement activities more broadly.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.