Improving transparency of deep neural inference process

Kuwajima, Hiroshi; Tanaka, Masayuki; Okutomi, Masatoshi

doi:10.1007/s13748-019-00179-x

Cited by 16 publications

(7 citation statements)

References 28 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Secondly, if many differently trained CNNs yield dissimilar results, how to produce a canonical comparer out of them? Alternatively, if using any of such CNNs for the task, how to audit their results [66] ? Thirdly, as reported by Nguyen et al [67] , wellperforming CNNs can lead to aberrant results that are misleading, even if they are produced with an almost-full ( > 99% ) confidence.…”

Section: Linear Feature Matching For Image Processingmentioning

confidence: 99%

A survey on matching strategies for boundary image comparison and evaluation

López-Molina

Marco-Detchart

Bustince

et al. 2021

Pattern Recognition

View full text Add to dashboard Cite

Section: Linear Feature Matching For Image Processingmentioning

confidence: 99%

A survey on matching strategies for boundary image comparison and evaluation

López-Molina

Marco-Detchart

Bustince

et al. 2021

Pattern Recognition

View full text Add to dashboard Cite

“…Further, interpretability is also useful for performance improvement, debugging during training, and validating of training results. Developers can understand the internal behavior of a trained NN to train higher performance models [45]. For example, a developer can visualize an NN's focus points for an incorrect inference and understand what was wrong, before additional training data is collected according to the analysis.…”

Section: Verification Of Machine Learning Modelsmentioning

confidence: 99%

Engineering problems in machine learning systems

Kuwajima

Yasuoka²,

Nakae³

2020

Mach Learn

Self Cite

View full text Add to dashboard Cite

Fatal accidents are a major issue hindering the wide acceptance of safety-critical systems that employ machine learning and deep learning models, such as automated driving vehicles. In order to use machine learning in a safety-critical system, it is necessary to demonstrate the safety and security of the system through engineering processes. However, thus far, no such widely accepted engineering concepts or frameworks have been established for these systems. The key to using a machine learning model in a deductively engineered system is decomposing the data-driven training of machine learning models into requirement, design, and verification, particularly for machine learning models used in safety-critical systems. Simultaneously, open problems and relevant technical fields are not organized in a manner that enables researchers to select a theme and work on it. In this study, we identify, classify, and explore the open problems in engineering (safety-critical) machine learning systems -that is, in terms of requirement, design, and verification of machine learning models and systems -as well as discuss related works and research directions, using automated driving vehicles as an example. Our results show that machine learning models are characterized by a lack of requirements specification, lack of design specification, lack of interpretability, and lack of robustness. We also perform a gap analysis on a conventional system quality standard SQuARE with the characteristics of machine learning models to study quality models for machine learning systems. We find that a lack of requirements specification and lack of robustness have the greatest impact on conventional quality models.Preprint. Work in progress.

show abstract

“…A quality sub-characteristic Operability has a measure Monitoring capability. Explainable AI (XAI) [8] is a rapid growing area in the artificial intelligence research, and techniques to explain or interpret ML components are proposed in recent years [13], [14]. They can be used for monitoring capacities for ML-based AI systems, but XAI research is still in the very early stage.…”

Section: Extension A1: Decomposition Of Evaluation Targetmentioning

confidence: 99%

Adapting SQuaRE for Quality Assessment of Artificial Intelligence Systems

Kuwajima

Ishikawa

2019

2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)

Self Cite

View full text Add to dashboard Cite

More and more software practitioners are tackling towards industrial applications of artificial intelligence (AI) systems, especially those based on machine learning (ML). However, many of existing principles and approaches to traditional systems do not work effectively for the system behavior obtained by training not by logical design. In addition, unique kinds of requirements are emerging such as fairness and explainability. To provide clear guidance to understand and tackle these difficulties, we present an analysis on what quality concepts we should evaluate for AI systems. We base our discussion on ISO/IEC 25000 series, known as SQuaRE, and identify how it should be adapted for the unique nature of ML and Ethics guidelines for trustworthy AI from European Commission. We thus provide holistic insights for quality of AI systems by incorporating the ML nature and AI ethics to the traditional software quality concepts.

show abstract

Improving transparency of deep neural inference process

Cited by 16 publications

References 28 publications

A survey on matching strategies for boundary image comparison and evaluation

A survey on matching strategies for boundary image comparison and evaluation

Engineering problems in machine learning systems

Adapting SQuaRE for Quality Assessment of Artificial Intelligence Systems

Contact Info

Product

Resources

About