Draw with Me: Human-in-the-Loop for Image Restoration

Weber, Thomas; Han, Zhiwei; Matthes, Stefan; Hußmann, Heinrich; Liu, Yuanting

doi:10.1007/978-3-030-58285-2_19

Cited by 11 publications

(25 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several papers for various ML application areas have been presented to improve the quality of ML methods where the role of humans to compensate for the deficiencies of ML methods is highlighted. Weber et al [ 6 ] proposed an Interactive Deep Image Prior (IDIP) which was based on Deep Neural Network and human-in-the-loop for image restoration. The role of humans was refining the output of the IDIP iteratively by providing some guidance specially missing semantics and controlling IDIP for better restoration as well as dealing with overfitting issue.…”

Section: Resultsmentioning

confidence: 99%

“…So it is the role of humans to control the ML process to get more desirable results. For example, in Weber et al [ 6 ], humans were more satisfied subjectively with the quality of the restored images generated by IDIP (human-in-the-loop approach) than DIP. This was tested using questionnaires and different groups of human experts.…”

Section: Resultsmentioning

confidence: 99%

“…Several papers have considered different domain human experts with different level of expertise to evaluate the performance of ML approaches. For example, in weber et al [ 6 ], the satisfaction of different groups of people with various characteristics and different levels of expertise when they were using the proposed ML method was considered as a measure to evaluate the performance of the ML method. In Yuksel et al [ 34 ], the satisfaction of blinded users from the ML approach to producing video description was introduced as an evaluation measure for the performance of the proposed HILML approach and evaluation results verified the outperformance of HILML than only ML approach from the view of users.…”

Section: Resultsmentioning

confidence: 99%

“…For example, In Yang et al [ 28 ], human expert interacted with the ML model iteratively until his desired results in terms of the F1 measure, precision and recall be achieved. In Weber et al [ 6 ], domain human experts collaborated in the learning process by providing new data samples iteratively to refine ML outputs in image reconstruction application. In Michalopoulos et al [ 42 ], human expert collaborated with the ML method by providing data labels interactively until a desirable result be achieved.…”

Section: Resultsmentioning

confidence: 99%

“…Therefore, the accuracy of these methods was confined [ 5 ]. Another example in Weber et al [ 6 ], found that an automatic image inpainting process by neural networks could not result in accurate and satisfactory outputs and only human knowledge could improve it. In addition, the difficulty of interpreting internal ML rules is mentioned as another drawback.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

Maadi

Khorshidi

Aickelin

2021

IJERPH

View full text Add to dashboard Cite

Objective: To provide a human–Artificial Intelligence (AI) interaction review for Machine Learning (ML) applications to inform how to best combine both human domain expertise and computational power of ML methods. The review focuses on the medical field, as the medical ML application literature highlights a special necessity of medical experts collaborating with ML approaches. Methods: A scoping literature review is performed on Scopus and Google Scholar using the terms “human in the loop”, “human in the loop machine learning”, and “interactive machine learning”. Peer-reviewed papers published from 2015 to 2020 are included in our review. Results: We design four questions to investigate and describe human–AI interaction in ML applications. These questions are “Why should humans be in the loop?”, “Where does human–AI interaction occur in the ML processes?”, “Who are the humans in the loop?”, and “How do humans interact with ML in Human-In-the-Loop ML (HILML)?”. To answer the first question, we describe three main reasons regarding the importance of human involvement in ML applications. To address the second question, human–AI interaction is investigated in three main algorithmic stages: 1. data producing and pre-processing; 2. ML modelling; and 3. ML evaluation and refinement. The importance of the expertise level of the humans in human–AI interaction is described to answer the third question. The number of human interactions in HILML is grouped into three categories to address the fourth question. We conclude the paper by offering a discussion on open opportunities for future research in HILML.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

Maadi

Khorshidi

Aickelin

2021

IJERPH

View full text Add to dashboard Cite

show abstract

Applications, Challenges, and Future Directions of Human-in-the-Loop Learning

Kumar,

Datta,

Singh

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Machine learning (ML) has become a popular technique for various automation tasks in the era of Industry 4.0, such as the analysis and synthesis of visual data such as images and videos, natural language and speech, financial data, and biomedical applications. However, ML-based automation techniques are facing difficulties like decision-making, thus incorporating user expertise into the system might be advantageous. The goal of adding human domain expertise with ML-based automation is to provide more accurate prediction models. Human-in-the-loop (HITL) systems that integrate human expertise with ML algorithms are becoming more and more common in various industries. However, there are a number of methodological, technical, and ethical difficulties with the development and application of HITL systems. This paper aims to explore the methodologies, challenges, and opportunities associated with HITL systems implementations. We also discuss a number of issues that must be resolved for HITL systems to be effective, including data quality, bias, and user engagement. Besides, we also explored several approaches that can be utilized to enhance the performance of HITL systems, such as active learning (AL), iterative ML, and reinforcement learning, as well as the current state of the art in HITL systems. We also selectively highlighted the advantages of HITL systems, such as their potential to increase decision-making process accountability and transparency by utilizing human experience to improve ML decision-making capability. The paper will be very useful for researchers, practitioners, and policymakers.

show abstract

Untrained Neural Network Priors for Inverse Imaging Problems: A Survey

Qayyum

Ilahi

Shamshad

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

In recent years, advancements in machine learning (ML) techniques, in particular, deep learning (DL) methods have gained a lot of momentum in solving inverse imaging problems, often surpassing the performance provided by hand-crafted approaches. Traditionally, analytical methods have been used to solve inverse imaging problems such as image restoration, inpainting, and superresolution. Unlike analytical methods for which the problem is explicitly defined and the domain knowledge is carefully engineered into the solution, DL models do not benefit from such prior knowledge and instead make use of large datasets to predict an unknown solution to the inverse problem. Recently, a new paradigm of training deep models using a single image, named untrained neural network prior (UNNP) has been proposed to solve a variety of inverse tasks, e.g., restoration and inpainting. Since then, many researchers have proposed various applications and variants of UNNP. In this paper, we present a comprehensive review of such studies and various UNNP applications for different tasks and highlight various open research problems which require further research.

show abstract

Draw with Me: Human-in-the-Loop for Image Restoration

Cited by 11 publications

References 3 publications

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

Applications, Challenges, and Future Directions of Human-in-the-Loop Learning

Untrained Neural Network Priors for Inverse Imaging Problems: A Survey

Contact Info

Product

Resources

About