Gestalt Principles in Multimodal Data Representation

Ringdahl

Fredriksson

2022

SASBE

PurposeThe noise and dust particles caused by the construction transport are by most stakeholders experienced as disturbing. The purpose of this study is to explore how sonification can support visualization in construction planning to decrease construction transport disturbances.Design/methodology/approachThis paper presents an interdisciplinary research project, combining research on construction logistics, internet of things and sonification. First, a data recording device, including sound, particle, temperature and humidity sensors, was implemented and deployed in a development project. Second, the collected data were used in a sonification design, which was, third, evaluated with potential users.FindingsThe results showed that the low-cost sensors used could capture “good enough” data, and that the use of sonification for representing these data is interesting and a possible useful tool in urban and construction transport planning.Research limitations/implicationsThere is a need to further evolve the sonification design and better communicate the aim of the sounds used to potential users. Further testing is also needed.Practical implicationsThis study introduces new ideas of how to support visualization with sonification planning the construction work and its impact on the vicinity of the site. Currently, urban planning and construction planning focus on visualizing the final result, with little focus on how to handle disturbances during the construction process.Originality/valueShowing the potentials of using low-cost sensor data in sonification, and using sonification together with visualization, is the result of a novel interdisciplinary research area combination.

Section: Sonification For Data Representationmentioning

confidence: 99%

“…, 2016). By adding sonification to visualization, more effective data representations can be designed (Rosli and Cabrera, 2015). Therefore, audiovisual analytics have the potential to efficiently provide information to a user (Rind et al.…”

Section: Introductionmentioning

confidence: 99%

Measurement and sonification of construction site noise and particle pollution data

Ringdahl

Fredriksson

2022

SASBE

“…the ears) providing additional input and further information [11,34]. By combining the visual and the aural modalities it should be possible to design more effective and efficient multimodal visual representations compared to when using visual stimuli alone [31].…”

Section: Introductionmentioning

confidence: 99%

Sonification supports perception of brightness contrast

J Multimodal User Interfaces

2019

In complex visual representations, there are several possible challenges for the visual perception that might be eased by adding sound as a second modality (i.e. sonification). It was hypothesized that sonification would support visual perception when facing challenges such as simultaneous brightness contrast or the Mach band phenomena. This hypothesis was investigated with an interactive sonification test, yielding objective measures (accuracy and response time) as well as subjective measures of sonification benefit. In the test, the participant's task was to mark the vertical pixel line having the highest intensity level. This was done in a condition without sonification and in three conditions where the intensity level was mapped to different musical elements. The results showed that there was a benefit of sonification, with higher accuracy when sonification was used compared to no sonification. This result was also supported by the subjective measurement. The results also showed longer response times when sonification was used. This suggests that the use and processing of the additional information took more time, leading to longer response times but also higher accuracy. There were no differences between the three sonification conditions.

“…However, they could be addressed by adding sound as a complementary modality to the visual representation. The combination of the visual and the aural modalities should make it possible to design more effective multimodal visual representations, as compared to when using visual stimuli alone (Rosli and Cabrera 2015).…”

Section: Introductionmentioning

confidence: 99%

Musical sonification supports visual discrimination of color intensity

Behaviour & Information Technology

2019

Visual representations of data introduce several possible challenges for the human visual perception system in perceiving brightness levels. Overcoming these challenges might be simplified by adding sound to the representation. This is called sonification. As sonification provides additional information to the visual information, sonification could be useful in supporting the visual perception. In the present study, usefulness (in terms of accuracy and response time) of sonification was investigated with an interactive sonification test. In the test, participants were asked to identify the highest brightness level in a monochrome visual representation. The task was performed in four conditions, one with no sonification and three with different sonification settings. The results show that sonification is useful, as measured by higher task accuracy, and that the participant's musicality facilitates the use of sonification with better performance when sonification was used. The results were also supported by subjective measurements, where participants reported an experienced benefit of sonification.