Combining audio and visual displays to highlight temporal and spatial seismic patterns

Paté, Arthur; Farge, Gaspard; Holtzman, B. K.; Barth, Anna; Poli, Piero; Boschi, Lapo; Karlstrom, L.

doi:10.1007/s12193-021-00378-8

Cited by 3 publications

(8 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The earth science cluster in our survey data holds eleven publications covering topics that range from oceanographic data [SAR22], wildfires [HK22], hurricanes [Bal15], and climate change [Bea11], to sonophenology [NRL*12], seismology [HCTP14,WW15,MMU16,PFH*22], and geospatial data displays [BF12,GDAS*18].…”

Section: Categorization and Resultsmentioning

confidence: 99%

“…Inclusion in this category requires the designer to put similar weight on the exploration and presentation of the data. To present a few examples, Paté et al [PFH*22] demonstrate their audiovisual display idiom with three seismic data sets, where the sonification methods are adapted to the specific properties of each data set. A multi‐scale audification method is presented where different speed factors are used depending on the size of each dataset, and different sound designs are used to highlight specific properties of the data.…”

Section: Categorization and Resultsmentioning

confidence: 99%

“…Only seven of them combine parameter mapping sonification with other techniques such as audification [MMU16,HPDW23], earcons [HAR16,NSC16], or auditory icons [BBV16, RFK*15, PBM15]. The technique of audification is part of the database three times on its own [PFH*22, HCTP14,WW15], and once in combination with earcons in a second design of a paper [PFH*22]. Yang and Hermann are the only authors employing model‐based sonification with their Mode Explorer design [YH17].…”

Section: Categorization and Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Open Your Ears and Take a Look: A State‐of‐the‐Art Report on the Integration of Sonification and Visualization

Enge,

Elmquist,

Caiola

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

The research communities studying visualization and sonification for data display and analysis share exceptionally similar goals, essentially making data of any kind interpretable to humans. One community does so by using visual representations of data, and the other community employs auditory (non‐speech) representations of data. While the two communities have a lot in common, they developed mostly in parallel over the course of the last few decades. With this STAR, we discuss a collection of work that bridges the borders of the two communities, hence a collection of work that aims to integrate the two techniques into one form of audiovisual display, which we argue to be “more than the sum of the two.” We introduce and motivate a classification system applicable to such audiovisual displays and categorize a corpus of 57 academic publications that appeared between 2011 and 2023 in categories such as reading level, dataset type, or evaluation system, to mention a few. The corpus also enables a meta‐analysis of the field, including regularly occurring design patterns such as type of visualization and sonification techniques, or the use of visual and auditory channels, showing an overall diverse field with different designs. An analysis of a co‐author network of the field shows individual teams without many interconnections. The body of work covered in this STAR also relates to three adjacent topics: audiovisual monitoring, accessibility, and audiovisual data art. These three topics are discussed individually in addition to the systematically conducted part of this research. The findings of this report may be used by researchers from both fields to understand the potentials and challenges of such integrated designs while hopefully inspiring them to collaborate with experts from the respective other field.

show abstract

Section: Categorization and Resultsmentioning

confidence: 99%

Section: Categorization and Resultsmentioning

confidence: 99%

Section: Categorization and Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Open Your Ears and Take a Look: A State‐of‐the‐Art Report on the Integration of Sonification and Visualization

Enge,

Elmquist,

Caiola

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…The database was divided into two independent and relatively equal parts for the training (recording index 1-10,000) and testing (10,837), without exclusion or stratification of ECG pathology, amplitude, or quality.…”

Section: Training and Testing In A Pc Simulation Platform 221 Ecg Dat...mentioning

confidence: 99%

“…In recent years, the advantages of sonification were demonstrated in the auditory representation of data from various fields, such as medicine, sports, financial markets, quantum physics, meteorology, situational monitoring implementations, etc. [ 6 , 7 , 8 , 9 , 10 , 11 , 12 ]. The sonification space was defined as a useful reference system for sonification tasks [ 13 ].…”

Section: Introductionmentioning

confidence: 99%

Application of Convolutional Neural Network for Decoding of 12-Lead Electrocardiogram from a Frequency-Modulated Audio Stream (Sonified ECG)

Krasteva,

Iliev,

Tabakov

2024

Sensors

View full text Add to dashboard Cite

Research of novel biosignal modalities with application to remote patient monitoring is a subject of state-of-the-art developments. This study is focused on sonified ECG modality, which can be transmitted as an acoustic wave and received by GSM (Global System for Mobile Communications) microphones. Thus, the wireless connection between the patient module and the cloud server can be provided over an audio channel, such as a standard telephone call or audio message. Patients, especially the elderly or visually impaired, can benefit from ECG sonification because the wireless interface is readily available, facilitating the communication and transmission of secure ECG data from the patient monitoring device to the remote server. The aim of this study is to develop an AI-driven algorithm for 12-lead ECG sonification to support diagnostic reliability in the signal processing chain of the audio ECG stream. Our methods present the design of two algorithms: (1) a transformer (ECG-to-Audio) based on the frequency modulation (FM) of eight independent ECG leads in the very low frequency band (300–2700 Hz); and (2) a transformer (Audio-to-ECG) based on a four-layer 1D convolutional neural network (CNN) to decode the audio ECG stream (10 s @ 11 kHz) to the original eight-lead ECG (10 s @ 250 Hz). The CNN model is trained in unsupervised regression mode, searching for the minimum error between the transformed and original ECG signals. The results are reported using the PTB-XL 12-lead ECG database (21,837 recordings), split 50:50 for training and test. The quality of FM-modulated ECG audio is monitored by short-time Fourier transform, and examples are illustrated in this paper and supplementary audio files. The errors of the reconstructed ECG are estimated by a popular ECG diagnostic toolbox. They are substantially low in all ECG leads: amplitude error (quartile range RMSE = 3–7 μV, PRD = 2–5.2%), QRS detector (Se, PPV > 99.7%), P-QRS-T fiducial points’ time deviation (<2 ms). Low errors generalized across diverse patients and arrhythmias are a testament to the efficacy of the developments. They support 12-lead ECG sonification as a wireless interface to provide reliable data for diagnostic measurements by automated tools or medical experts.

show abstract

Scale as Ratio in Time

Suteanu

2022

Scale

View full text Add to dashboard Cite

Combining audio and visual displays to highlight temporal and spatial seismic patterns

Cited by 3 publications

References 32 publications

Open Your Ears and Take a Look: A State‐of‐the‐Art Report on the Integration of Sonification and Visualization

Open Your Ears and Take a Look: A State‐of‐the‐Art Report on the Integration of Sonification and Visualization

Application of Convolutional Neural Network for Decoding of 12-Lead Electrocardiogram from a Frequency-Modulated Audio Stream (Sonified ECG)

Scale as Ratio in Time

Contact Info

Product

Resources

About