Design and Validation of an Explainable Fuzzy Beer Style Classifier

Alonso, José M.; Castiello, Ciro; Magdalena, Luis; Mencar, Corrado

doi:10.1007/978-3-030-71098-9_6

Cited by 5 publications

(3 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A usual strategy to generate explanations for decisions made by a black-box machine learning model, such as a deep learning model, is to build a surrogate model based on more expressive machine learning algorithms, such as the aforementioned decision rules [10,14], decision trees [12,1], or linear models [13]. The surrogate model is trained on the same data set as the black-box model to be explained or on new data points classified by that same model.…”

Section: Contribution and Plan Of This Papermentioning

confidence: 99%

Machine Learning Explainability Through Comprehensible Decision Trees

Blanco-Justicia

Domingo-Ferrer

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The role of decisions made by machine learning algorithms in our lives is ever increasing. In reaction to this phenomenon, the European General Data Protection Regulation establishes that citizens have the right to receive an explanation on automated decisions affecting them. For explainability to be scalable, it should be possible to derive explanations in an automated way. A common approach is to use simpler, more intuitive decision algorithms to build a surrogate model of the black-box model (for example a deep learning algorithm) used to make a decision. Yet, there is a risk that the surrogate model is too large for it to be really comprehensible to humans. We focus on explaining black-box models by using decision trees of limited size as a surrogate model. Specifically, we propose an approach based on microaggregation to achieve a trade-off between comprehensibility and representativeness of the surrogate model on the one side and privacy of the subjects used for training the black-box model on the other side.

show abstract

Section: Contribution and Plan Of This Papermentioning

confidence: 99%

Machine Learning Explainability Through Comprehensible Decision Trees

Blanco-Justicia

Domingo-Ferrer

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…This can be general ("when do you do X") or specific to a given action. An example of a specific explanation is a natural language explanation for a classification in the ML space (Alonso et al, 2018) or the generation of a summary of "when do you do X?" type questions in natural language to explain the actions of an agent (Hayes & Shah, 2017).…”

Section: Interpretable/explainable Decision-making Processes Of Rlmentioning

confidence: 99%

“…We refer to examples that codify the decision process of the model as rules (Liu et al, 2018), as code blocks (Verma et al, 2018), or through natural language. Alonso et al (2018) shows an example of justifying classifications with a textual explanation of the choice made by a decision tree, which could in turn be transferred to RL applications. Other examples for providing an introspection into LLMs with textual explanations are Zini and Awad (2022) and Xu et al (2023).…”

Section: Approach: Explainabilitymentioning

confidence: 99%

Human-in-the-Loop Reinforcement Learning: A Survey and Position on Requirements, Challenges, and Opportunities

Retzlaff,

Das,

Wayllace

et al. 2024

jair

View full text Add to dashboard Cite

Artificial intelligence (AI) and especially reinforcement learning (RL) have the potential to enable agents to learn and perform tasks autonomously with superhuman performance. However, we consider RL as fundamentally a Human-in-the-Loop (HITL) paradigm, even when an agent eventually performs its task autonomously. In cases where the reward function is challenging or impossible to define, HITL approaches are considered particularly advantageous. The application of Reinforcement Learning from Human Feedback (RLHF) in systems such as ChatGPT demonstrates the effectiveness of optimizing for user experience and integrating their feedback into the training loop. In HITL RL, human input is integrated during the agent’s learning process, allowing iterative updates and fine-tuning based on human feedback, thus enhancing the agent’s performance. Since the human is an essential part of this process, we argue that human-centric approaches are the key to successful RL, a fact that has not been adequately considered in the existing literature. This paper aims to inform readers about current explainability methods in HITL RL. It also shows how the application of explainable AI (xAI) and specific improvements to existing explainability approaches can enable a better human-agent interaction in HITL RL for all types of users, whether for lay people, domain experts, or machine learning specialists. Accounting for the workflow in HITL RL and based on software and machine learning methodologies, this article identifies four phases for human involvement for creating HITL RL systems: (1) Agent Development, (2) Agent Learning, (3) Agent Evaluation, and (4) Agent Deployment. We highlight human involvement, explanation requirements, new challenges, and goals for each phase. We furthermore identify low-risk, high-return opportunities for explainability research in HITL RL and present long-term research goals to advance the field. Finally, we propose a vision of human-robot collaboration that allows both parties to reach their full potential and cooperate effectively.

show abstract