Transformers and the representation of biomedical background knowledge

Wysocki, Oskar; Zhou, Zili; O'Regan, Paul; Ferreira, Deborah; Wysocka, Magdalena; Landers, Dónal; Freitas, André Victor Lucci

doi:10.48550/arxiv.2202.02432

Cited by 2 publications

(2 citation statements)

References 23 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to recent surveys in Explainable AI (XAI) (i.a. [7,8,9,10,11,12,13]), XAI in the field of genomics [14,15,16] and medicine [17,18,19,20,21,22], we highlight a much more specific subfield, aiming to link explainability and biological interpretability. The survey is restricted to the context of multi-omics based DL in cancer biology, excluding papers from the computer-vision subarea.…”

Section: Feature Spacementioning

confidence: 99%

A systematic review of biologically-informed deep learning models for cancer: fundamental trends for encoding and interpreting oncology data

Wysocka¹,

Wysocki²,

Zufferey³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper we provide a structured literature analysis focused on Deep Learning (DL) models used to support inference in cancer biology with a particular emphasis on multi-omics analysis. The work focuses on how existing models address the need for better dialogue with prior knowledge, biological plausibility and interpretability, fundamental properties in the biomedical domain. We discuss the recent evolutionary arch of DL models in the direction of integrating prior biological relational and network knowledge to support better generalisation (e.g. pathways or Protein-Protein-Interaction networks) and interpretability. This represents a fundamental functional shift towards models which can integrate mechanistic and statistical inference aspects. We discuss representational methodologies for the integration of domain prior knowledge in such models. The paper also provides a critical outlook into contemporary methods for explainability and interpretabiltiy. This analysis points in the direction of a convergence between encoding prior knowledge and improved interpretability.

show abstract

Section: Feature Spacementioning

confidence: 99%

A systematic review of biologically-informed deep learning models for cancer: fundamental trends for encoding and interpreting oncology data

Wysocka¹,

Wysocki²,

Zufferey³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition to recent surveys in Explainable AI (XAI) (i.a. [11,12,13,14,15,16,17]), XAI in the field of genomics [18,19,20] and medicine [21,22,23,24,25,26], we highlight a much more specific subfield, aiming to link explainability and biological interpretability. The systematic review is restricted to the context of multi-omics based DL in cancer biology, excluding papers from the computer-vision subarea.…”

Section: Introductionmentioning

confidence: 99%

A systematic review of biologically-informed deep learning models for cancer: fundamental trends for encoding and interpreting oncology data

Wysocka

Wysocki

Zufferey

et al. 2022

Preprint

View full text Add to dashboard Cite

Background There is an increasing interest in the use of Deep Learning (DL) based methods as a supporting analytical framework in oncology. However, most direct applications of DL will deliver models with limited transparency and explainability, which constrain their deployment in biomedical settings. Methods This systematic review discusses DL models used to support inference in cancer biology with a particular emphasis on multi-omics analysis. It focuses on how existing models address the need for better dialogue with prior knowledge, biological plausibility and interpretability, fundamental properties in the biomedical domain. For this, we retrieved and analyzed 42 studies focusing on emerging architectural and methodological advances, the encoding of biological domain knowledge and the integration of explainability methods. Results We discuss the recent evolutionary arch of DL models in the direction of integrating prior biological relational and network knowledge to support better generalisation (e.g. pathways or Protein-Protein-Interaction networks) and interpretability. This represents a fundamental functional shift towards models which can integrate mechanistic and statistical inference aspects. We introduce a concept of bio-centric interpretability and according to its taxonomy, we discuss representational methodologies for the integration of domain prior knowledge in such models. Conclusions The paper provides a critical outlook into contemporary methods for explainability and interpretabiltiy used in DL for cancer. The analysis points in the direction of a convergence between encoding prior knowledge and improved interpretability. We introduce bio-centric interpretability which is an important step towards formalisation of biological interpretability of DL models and developing methods that are less problem- or application-specific.

show abstract

Transformers and the representation of biomedical background knowledge

Cited by 2 publications

References 23 publications

A systematic review of biologically-informed deep learning models for cancer: fundamental trends for encoding and interpreting oncology data

A systematic review of biologically-informed deep learning models for cancer: fundamental trends for encoding and interpreting oncology data

A systematic review of biologically-informed deep learning models for cancer: fundamental trends for encoding and interpreting oncology data

Contact Info

Product

Resources

About