Incorporating Prior Domain Knowledge into Deep Neural Networks

Muralidhar, Nikhil; Islam, Mohammad Raihanul; Marwah, Manish; Karpatne, Anuj; Ramakrishnan, Naren

doi:10.1109/bigdata.2018.8621955

Cited by 96 publications

(70 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Approximation and monotonicity constraints were also incorporated into the loss function of a NN [224] for predicting oxygen solubility in water.…”

Section: Physics Based Regularizationmentioning

confidence: 99%

“…A simple example could be expecting a model that has been trained for classifying the predecessors of human beings (e.g., monkeys, apes, and gorillas) to perform well on classifying human beings. The hybrid models have portrayed excellent extrapolability [133], [200], [215], [224], [225], [253].…”

Section: ) Extrapolabilitymentioning

confidence: 99%

“…Training accurate hybrid models for regression problems with a low amount of data is a fertile field for future research. The prior works [133], [200], [215], [224], [225] has shown that hybrid models considerably outperformed both MB and ML in terms of interpolability and extrapolability in a variety of applications with limited data. However, the application problems in these existing work are limited to simple systems.…”

Section: Regression Problems With Low Amount Of Datamentioning

confidence: 99%

See 2 more Smart Citations

Driven by Data or Derived Through Physics? A Review of Hybrid Physics Guided Machine Learning Techniques With Cyber-Physical System (CPS) Focus

2020

View full text Add to dashboard Cite

A multitude of cyber-physical system (CPS) applications, including design, control, diagnosis, prognostics, and a host of other problems, are predicated on the assumption of model availability. There are mainly two approaches to modeling: Physics/Equation based modeling (Model-Based, MB) and Machine Learning (ML). Recently, there is a growing consensus that ML methodologies relying on data need to be coupled with prior scientific knowledge (or physics, MB) for modeling CPS. We refer to the paradigm that combines MB approaches with ML as hybrid learning methods. Hybrid modeling (HB) methods is a growing field within both the ML and scientific communities, and are recognized as an important emerging but nascent area of research. Recently, several works have attempted to merge MB and ML models for the complete exploitation of their combined potential. However, the research literature is scattered and unorganized. So, we make a meticulous and systematic attempt at organizing and standardizing the methods of combining ML and MB models. In addition to that, we outline five metrics for the comprehensive evaluation of hybrid models. Finally, we conclude by shedding some light on the challenges of hybrid models, which we, as a research community, should focus on for harnessing the full potential of hybrid models. An additional feature of this survey is that the hybrid modeling work has been discussed with a focus on modeling cyber-physical systems. INDEX TERMS Cyber-physical systems, deep learning, deep neural networks, hybrid models, model-based, machine learning, physics guided, physics informed, physics prior, theory guided.

show abstract

“…Approximation and monotonicity constraints were also incorporated into the loss function of a NN [224] for predicting oxygen solubility in water.…”

Section: Physics Based Regularizationmentioning

confidence: 99%

Section: ) Extrapolabilitymentioning

confidence: 99%

Section: Regression Problems With Low Amount Of Datamentioning

confidence: 99%

See 1 more Smart Citation

Driven by Data or Derived Through Physics? A Review of Hybrid Physics Guided Machine Learning Techniques With Cyber-Physical System (CPS) Focus

2020

View full text Add to dashboard Cite

show abstract

“…There are two potential strategies to address the challenge. One strategy is to incorporate domain knowledge into the design of model structures or model components [144], [145] so that the model is aware of the underlying physics. The other strategy is to explicitly model the spatial structural constraints (e.g., from computational topology) and build it into the backbone of model representation.…”

Section: B Model Transparency and Interpretabilitymentioning

confidence: 99%

Spatial Structured Prediction Models: Applications, Challenges, and Techniques

Jiang

2020

IEEE Access

View full text Add to dashboard Cite

Spatial structure patterns are prevalent in many real-world data and applications. For example, in biochemistry, the geometric topology of a molecular surface indicates protein functions; in hydrology, irregular geographic terrains and topography on the Earth's surface control water flows and distributions; in civil engineering, wetland parcels in remote sensing imagery are often made up of contiguous patches. Spatial structured prediction aims to learn a prediction model whose input and output data contain a spatial structure. Modeling spatial structural information in prediction models is critical for interdisciplinary applications due to two reasons. First, explicit spatial structural information often indicates the underlying physical process, and thus enhances model interpretability. Second, spatial structural constraints also have positive side-effects of enhancing model robustness against noise and obstacles and regularizing model learning when training labels are limited. However, spatial structured prediction also poses several unique challenges, such as the existence of implicit spatial structure in continuous space, structural complexity in geometric topology, and high computational costs. Over the years, various techniques have been proposed for spatial structured prediction in different applications. This paper aims to provide an overview of the spatial structured prediction problem. We provide a taxonomy of techniques based on the underlying approaches. We also discuss several future research directions. The paper can potentially not only help interdisciplinary researchers find relevant techniques but also help machine learning researchers identify new research opportunities.

show abstract

“…Classification and regression are essential tasks in machine learning and data analytics. Although very powerful artificial intelligence tools meet these demands, the integration of prior knowledge into such approaches requires specific solutions (Ajraoui, Parra‐Robles, & Wild, 2013; d. Bezenac, Pajot, & Gallinari, ; Du, Feldman, Li, & Jin, ; Duivesteijn & Feelders, ; Finsy, de Groen, Deriemaeker, Gelad, & Joosten, 1992; Moreno, Regueiro, Iglesias, & Barro, ; Muralidhar, Islam, Marwah, Karpatne, & Ramakrishnan, ; Thompson & Kramer, ; Yu, Jan, Simoff, & Debenham, ). Here, we want to establish a formal model to ensure a monotonic dependency between input vectors and their associated output states (J.‐R.…”

Section: Introductionmentioning

confidence: 99%

A class of monotone kernelized classifiers on the basis of the Choquet integral

2020

View full text Add to dashboard Cite

The key property of monotone classifiers is that increasing (decreasing) input values lead to increasing (decreasing) the output value. Preserving monotonicity for a classifier typically requires many constraints to be respected by modelling approaches such as artificial intelligence techniques. The type of constraints strongly depends on the modelling assumptions. Of course, for sophisticated models, such conditions might be very complex. In this study, we present a new family of kernels that we call it Choquet kernels. Henceforth, it allows for employing popular kernel‐based methods, such as support vector machines. Instead of a naïve approach with exponential computational complexity, we propose an equivalent formulation with quadratic time in the number of attributes. Furthermore, because coefficients derived from kernel solutions are not necessarily monotone in the dual form, different approaches are proposed to monotonize coefficients. Finally, experiments illustrate beneficial properties of the Choquet kernels.

show abstract

Incorporating Prior Domain Knowledge into Deep Neural Networks

Cited by 96 publications

References 13 publications

Driven by Data or Derived Through Physics? A Review of Hybrid Physics Guided Machine Learning Techniques With Cyber-Physical System (CPS) Focus

Driven by Data or Derived Through Physics? A Review of Hybrid Physics Guided Machine Learning Techniques With Cyber-Physical System (CPS) Focus

Spatial Structured Prediction Models: Applications, Challenges, and Techniques

A class of monotone kernelized classifiers on the basis of the Choquet integral

Contact Info

Product

Resources

About