Ablate, Variate, and Contemplate: Visual Analytics for Discovering Neural Architectures

Abstract

A B C Fig. 1: A screenshot of the REMAP system. In the model overview, section A, a visual overview of the set of sampled models is shown. Darkness of circles encodes performance of the models, and radius encodes the number of parameters. In the model drawer, section B, users can save models during their exploration for comparison or to return to later. In section C, four tabs help the user explore the model space and generate new models. The Generate Models tab, currently selected, allows for users to create new models via ablations, variations, or handcrafted templates.Abstract-Deep learning models require the configuration of many layers and parameters in order to get good results. However, there are currently few systematic guidelines for how to configure a successful model. This means model builders often have to experiment with different configurations by manually programming different architectures (which is tedious and time consuming) or rely on purely automated approaches to generate and train the architectures (which is expensive). In this paper, we present Rapid Exploration of Model Architectures and Parameters, or REMAP, a visual analytics tool that allows a model builder to discover a deep learning model quickly via exploration and rapid experimentation of neural network architectures. In REMAP, the user explores the large and complex parameter space for neural network architectures using a combination of global inspection and local experimentation. Through a visual overview of a set of models, the user identifies interesting clusters of architectures. Based on their findings, the user can run ablation and variation experiments to identify the effects of adding, removing, or replacing layers in a given architecture and generate new models accordingly. They can also handcraft new models using a simple graphical interface. As a result, a model builder can build deep learning models quickly, efficiently, and without manual programming. We inform the design of REMAP through a design study with four deep learning model builders. Through a use case, we demonstrate that REMAP allows users to discover performant neural network architectures efficiently using visual exploration and user-defined semi-automated searches through the model space.

Deep Learning Based Emotion Recognition and Visualization of Figural Representation

“… Murugesan et al (2019) conducted a comparative analysis on the performance of deep learning model in visualization and interaction and selected two real cases for the preliminary assessment to show that experts could make wise decisions on the effectiveness of different types of models. Cashman et al (2019) investigated visual analysis through neural architecture and introduced the Fast Exploration of Model Architectures and Parameters. This visual analysis tool allowed the model builder to quickly discover the deep learning model through the exploration and rapid experiment of neural network architecture.…”

“…Cashman et al (2019) investigated visual analysis through neural architecture and introduced the Fast Exploration of Model Architectures and Parameters. This visual analysis tool allowed the model builder to quickly discover the deep learning model through the exploration and rapid experiment of neural network architecture.…”

The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations

“…Cashman et al [CPCS20] researched the rapid exploration of model architectures and parameters. To this end, they developed a VA tool that allows a model developer to discover a DL model immediately via exploration as well as rapid deployment and examination of NN architectures.…”

NAS-Navigator: Visual Steering for Explainable One-Shot Deep Neural Network Synthesis

“…TreePOD [43] provides an interface to manage the trade-off between accuracy and interpretability of different existing machine learning models that would fit a particular dataset. REMAP [12] allows interactive convolutional neural network architecture search starting with a few pre-trained models, which are used to identify potential architectures towards better accuracy. Besides designing neural networks, there also exist tools that allow interactive design and filtering of clustering techniques [13,31,46,53] and dimension reduction [6,14,25,37,47].…”

“…As a result, these networks are very difficult to generalize because of the very high hardware equipment demands and associated costs. There also exist some visual analytics tools to include humans in the loop along with automated search techniques [12]. However, they still require pre-trained candidate models to start the search process.…”