A Deep Learning Approach to Selecting Representative Time Steps for Time-Varying Multivariate Data

DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualization

“…They then employed vector quantization to the high-level features extracted from volume patches to generate the characteristic feature vector to support the hierarchical exploration of complex volumetric structures. Porter et al [117] leveraged an AE to encode each timestep of a time-varying volumetric dataset into a feature vector, which was then projected to an abstract 2D space for identifying representative timesteps. Their approach can naturally handle multivariate datasets using a multichannel input which previous works cannot.…”

“…He et al [64] designed ScalarGCN, a GNN-based solution for scalarvalue association analysis of volumes. ScalarGCN aims to [120] image pairs feature vectors Cheng et al [24] volume patch feature vector Porter et al [117] volume feature vector Tkachev et al [144] S4 local spatiotemporal patches feature vectors He et al [64] ScalarGCN Scalar-Graph with local and global connections feature vector per variable Han et al [48] FlowNet streamline or stream surface feature vector Han and Wang [52] SurfNet isosurface or stream surface node features Chu and Thuerey [26] flow patch pairs feature vectors Liu et al [103] density field feature vector Li and Shen [97] particle patch feature vector Zhu et al [183] feature position in initial scatterplot inferred feature position in new scatterplot learn the high-order topological structural relationships of multiple variables using a multilayer GCN with the selfattention mechanism. For vector field data and fluid simulation, Han et al [48] introduced FlowNet, which is an AE for learning the latent features of streamlines and stream surfaces implicitly.…”

“…The learned features are projected into a low-dimensional space to support clustering, filtering, and selection of representatives. Their voxel-based representation makes the solution generally applicable to any 3D data or their visual representations (e.g., [117]), albeit not necessarily efficient when the representations are sparse in the 3D domain. Han and Wang [52] designed SurfNet, a GCN-based solution for learning node and surface features for isosurfaces and stream surfaces.…”

“…A close look at the categorization under CNN shows that encoder+decoder is the most diverse, covering all five research tasks, while decoder covers two tasks (i.e., data [2](U-Net, deformable CNN), [47] [11](CNN vs. ResNet vs. U-Net), [15](U-Net), [27] (GAN) [49] [50](GAN), [55] (GAN) [35], [40](nested encoder+decoder), [45], [46], [53] [76](ESPCN), [106] (GAN), [126] [54] (GAN), [56] (GAN), [70] (GAN) [159] (FRVSR-Net), [161](EnhanceNet) [81](symmetric FCN), [84](AE), [103](residual AE) [162] (multi-pass GAN) [116](U-Net vs. U-Net+ResNet vs. DenseNet) [167] (ESRGAN, WGAN) [117](AE), [154](multi-stream CNN) [168] (GAN), [182] [158](U-Net, V-Net), [163], [164] MLP [57], [183](FCCNN) - [84], [109], [160] RNN [164](stacked LSTM) [163](LSTM) -GNN [52](GCN), [64](GCN) [131](GCN) -CNN+MLP [24], [26](Siamese) [63] [12] (GAN), [48](AE) [31], [62], [85] [97](Geo-CNN)…”

IsoExplorer: an isosurface-driven framework for 3D shape analysis of biomedical volume data

“…In addition, deep learning can be used for visual analysis between multiple volumes. Porter et al (2019) proposed an autoencoder model to learn a representation for each volume in a time-varying multivariate dataset and then leveraged them for representative time step selection. Han et al (2021) designed a GAN to enable exploration of multivariate time-varying data in variable selection and translation analysis.…”