PAVE: An In Situ Framework for Scientific Visualization and Machine Learning Coupling

Leventhal, Samuel; Kim, Mark; Pugmire, David

doi:10.1109/drbsd-549595.2019.00007

Cited by 4 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is a growing interest in new developments at the intersection of machine learning and scientific visualization (Bertini and Lalanne, 2009). Some contributions integrate visualization in certain steps of machine learning to customize training of models (Chen et al, 2017b;Li et al, 2018;Liu et al, 2019), while some integrate machine learning algorithms into visualization (Chalupa and Mikulka, 2018;Chauhan et al, 2020;Lasso et al, 2020;Leventhal et al, 2019;Tzeng and Ma, 2005). This work concentrates on the latter case, where the objective is to enhance the visualization process and gain better insight into the given data.…”

Section: Related Workmentioning

confidence: 99%

“…Given these observations, there is a sizeable amount of work on integrating machine learning algorithms into visualization (Chalupa and Mikulka, 2018;Chauhan et al, 2020;Lasso et al, 2020;Leventhal et al, 2019;Tzeng and Ma, 2005). As we will also discuss in Section 2, related work suggests that there is a benefit from integrating machine learning models such as those for classification and segmentation into different parts of visualization, including the earlier given example of fluid data (Guo et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Toward Data-Driven Filters in Paraview

Maharjan

Zaspel

2022

J Flow Vis Image Proc

View full text Add to dashboard Cite

Scientific visualization is a crucial part of many computational workflows in science and engineering. A given visualization is obtained by applying a series of filters or transformations to input data and finally mapping the filtered data to a graphical representation. So far, such filters mostly use deterministic algorithms to process the data. In this work, we aim at extending this methodology toward data-driven filters, filters that expose the abilities of pretrained machine learning models to the visualization system. The use of such data-driven filters can be of particular interest in fields like segmentation, classification, and surrogate models, where machine learning models often outperform existing algorithmic approaches. The objective of this work is to give a starting point for developments toward data-driven filters in multipurpose visualization tools. To this end, we propose, develop, and provide a first software prototype that combines the standard visualization tool ParaView with the state-of-the-art deep learning framework PyTorch in order to expose machine learning models as data-driven filters. Several simple use cases for segmentation and classification filters are presented, showing the technical applicability of our approach. The resulting prototype is provided as open source to facilitate the future development of data-driven filters.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Toward Data-Driven Filters in Paraview

Maharjan

Zaspel

2022

J Flow Vis Image Proc

View full text Add to dashboard Cite

show abstract

“…As discussed in Section 1, most works have focused on introducing machine learning into some areas of scientific visualization to enhance the visualization process and gain better insights on the given data. For example, Leventhal et al [31] propose acceleration of path-tracing with neural networks, Chauhan et al [7] demonstrate a graphical user interface that uses machine learning based segmentation to enable region of interest selection, and Tzeng et al [14] present a visualization system that learns to extract and track features in flow simulation. Similarly, machine learning extensions [6,30] for segmentation have been developed for 3D Slicer [15], an open-source software application for medical image computing.…”

Section: Related Workmentioning

confidence: 99%

“…In Section 2, relevant work introducing different variants of such combination are addressed. Some have integrated visualization in certain steps of machine learning to customize training of models [9,32,33], while some have integrated machine learning algorithms into visualization [6,7,14,30,31]. This work is concerned with the latter case by introducing data-driven classification and segmentation filters in Paraview, pretrained using PyTorch.…”

Section: Introductionmentioning

confidence: 99%

Towards data-driven filters in Paraview

Maharjan¹,

Zaspel²

2021

Preprint

View full text Add to dashboard Cite

Recent progress in scientific visualization has expanded the scope of visualization from being merely a way of presentation to an analysis and discovery tool. A given visualization result is usually generated by applying a series of transformations or filters to the underlying data. Nowadays, such filters use deterministic algorithms to process the data. In this work, we aim at extending this methodology towards data-driven filters, thus filters that expose the abilities of pre-trained machine learning models to the visualization system. The use of such data-driven filters is of particular interest in fields like segmentation, classification, etc., where machine learning models regularly outperform existing algorithmic approaches. To showcase this idea, we couple Paraview, the well-known flow visualization tool, with PyTorch, a deep learning framework. Paraview is extended by plugins that allow users to load pre-trained models of their choice in the form of newly developed filters. The filters transform the input data by feeding it into the model and then provide the model's output as input to the remaining visualization pipeline. A series of simplistic use cases for segmentation and classification on image and fluid data is presented to showcase the technical applicability of such data-driven transformations in Paraview for future complex analysis tasks.

show abstract

“…In recent years, the scientific visualization community has seen an increased adoption of deep learning (Berger et al, 2018;Engel and Ropinski, 2020;Han et al, 2020;He et al, 2019;Hong et al, 2019;Leventhal et al, 2019;Weiss et al, 2019), including multiple research projects that consider vector field data (Guo et al, 2020;Han et al, 2018Jakob et al, 2020;Liu et al, 2019;Sahoo and Berger, 2021). With respect to exploratory Lagrangian-based particle advection schemes, the use of deep learning has not previously been studied to the best of our knowledge.…”

Section: Introductionmentioning

confidence: 99%

Exploratory Lagrangian-Based Particle Tracing Using Deep Learning

Han

Sane

Johnson

2022

J Flow Vis Image Proc

View full text Add to dashboard Cite

Time-varying vector fields produced by computational fluid dynamics simulations are often prohibitively large and pose challenges for accurate interactive analysis and exploration. To address these challenges, reduced Lagrangian representations have been increasingly researched as a means to improve scientific time-varying vector field exploration capabilities. This paper presents a novel deep neural network based particle tracing method to explore time-varying vector fields represented by Lagrangian flow maps. In our workflow, in situ processing is first utilized to extract Lagrangian flow maps, and deep neural networks then use the extracted data to learn flow field behavior. Using a trained model to predict new particle trajectories offers a fixed-small memory footprint and fast inference. To demonstrate and evaluate the proposed method, we perform an in-depth study of performance using a well-known analytical data set, the double gyre. Our study considers two flow map extraction strategies as well as the impact of the number of training samples and integration durations on efficacy, evaluates multiple sampling options for training and testing, and informs hyperparameter settings. Overall, we find our method requires a small fixed-memory footprint of 10.5 MB to encode a Lagrangian representation of a time-varying vector field while maintaining accuracy. For post hoc analysis, loading the trained model costs only two seconds, significantly reducing the burden of I/O when reading data for visualization. Moreover, our parallel implementation can infer one hundred locations for each of two thousand new pathlines across the entire temporal resolution in 1.3 seconds using one NVIDIA Titan RTX GPU.

show abstract

PAVE: An In Situ Framework for Scientific Visualization and Machine Learning Coupling

Cited by 4 publications

References 16 publications

Toward Data-Driven Filters in Paraview

Toward Data-Driven Filters in Paraview

Towards data-driven filters in Paraview

Exploratory Lagrangian-Based Particle Tracing Using Deep Learning

Contact Info

Product

Resources

About