Categorical representation learning: morphism is all you need

Sheshmani, Artan; You, Yi-Zhuang; Azizi, Ahmadreza

doi:10.1088/2632-2153/ac2c5d

“…In AWIE, the authors combined isometric embedding and attribute weighting, effectively mitigating dimensionality expansion and improving classification performance. Another novel approach to categorical representation learning was termed the 'categorifier' by [39]. The proposed solution addressed the challenge of improving representation learning beyond traditional set-theoretic methods.…”

Section: A Embedding Techniquesmentioning

confidence: 99%

Road Crash Injury Severity Prediction Using a Graph Neural Network Framework

Sattar,

Ishak,

Affendey

et al. 2024

IEEE Access

0

View full text Add to dashboard Cite

Crash severity prediction is a challenging research area, where the objective is to accurately assess the extent of severity of an injury resulting from road traffic accidents. The main aim of existing studies is to precisely assess the potential severity of crashes under diverse circumstances, such as weather conditions, vehicle attributes, road characteristics and layout, and traffic control factors. This effort aids authorities in establishing effective emergency response systems. The novelty and objective of our work involve contributing to this research area by employing a graph architecture to capture relationships among various crash records to uncover any hidden patterns that traditional ML models might overlook. The current study extends existing knowledge by leveraging Graph Neural Networks (GNN) and comparing their performance to popular ensemble-based models, which include Extreme Gradient Boosting (XGBoost) and Random Forest (RF) and Artificial Neural Networks (ANNs). Real data from the United Kingdom (UK) was employed to achieve our goal. All models underwent training using the training dataset, followed by performance evaluation using diverse metrics such as the accuracy, precision, recall, f1-score, Matthews Correlation Coefficient (MCC), confusion matrix, and computational cost on the test dataset. Overall, our proposed GNN-based model demonstrated better performance when compared to other models. Specifically, the GNN model outperformed all other models across all metrics. For instance, the accuracy stood at 85.55% for GNN as compared to 83.36%, 83.18%, and 83.27% for XGBoost, RF, and ANNs models, respectively. The GNN model assisted in identifying hidden patterns by considering non-linear relationships among crash records. Thus, the model had the potential to improve its ability to predict severe accidents, which could in turn significantly improve emergency response efforts and reduce the likelihood of severe accidents resulting in fatalities.

show abstract

“…This would yield a diagram of the form This approach can also be applied to more complex datasets than the ones we have studied in this paper using the categorical language and adapting the dissimilarity metrics. One example of this can be found in [SY21]. One thing to note about [SY21] is that the authors assume we have a vector representation of a certain category C. The downside of this approach is that one does not have a way to check how biased the vector representation is.…”

Section: Future Workmentioning

confidence: 99%

“…One example of this can be found in [SY21]. One thing to note about [SY21] is that the authors assume we have a vector representation of a certain category C. The downside of this approach is that one does not have a way to check how biased the vector representation is. With that in mind, one could apply this method to the semantic dataset and have metrics of how biased, or how much bias is tolerated by, the algorithms.…”

Section: Future Workmentioning

confidence: 99%

Exploring explainable AI: category theory insights into machine learning algorithms

Fabregat-Hernández,

Palanca,

Botti

2023

Mach. Learn.: Sci. Technol.

1

0

View full text Add to dashboard Cite

Explainable artificial intelligence (XAI) is a growing field that aims to increase the transparency and interpretability of machine learning (ML) models. The aim of this work is to use the categorical properties of learning algorithms in conjunction with the categorical perspective of the information in the datasets to give a framework for explainability. In order to achieve this, we are going to define the enriched categories, with decorated morphisms, L e a r n L e a r n , P a r a P a r a and M N e t M N e t of learners, parameterized functions, and neural networks over metric spaces respectively. The main idea is to encode information from the dataset via categorical methods, see how it propagates, and lastly, interpret the results thanks again to categorical (metric) information. This means that we can attach numerical (computable) information via enrichment to the structural information of the category. With this, we can translate theoretical information into parameters that are easily understandable. We will make use of different categories of enrichment to keep track of different kinds of information. That is, to see how differences in attributes of the data are modified by the algorithm to result in differences in the output to achieve better separation. In that way, the categorical framework gives us an algorithm to interpret what the learning algorithm is doing. Furthermore, since it is designed with generality in mind, it should be applicable in various different contexts. There are three main properties of category theory that help with the interpretability of ML models: formality, the existence of universal properties, and compositionality. The last property offers a way to combine smaller, simpler models that are easily understood to build larger ones. This is achieved by formally representing the structure of ML algorithms and information contained in the model. Finally, universal properties are a cornerstone of category theory. They help us characterize an object, not by its attributes, but by how it interacts with other objects. Thus, we can formally characterize an algorithm by how it interacts with the data. The main advantage of the framework is that it can unify under the same language different techniques used in XAI. Thus, using the same language and concepts we can describe a myriad of techniques and properties of ML algorithms, streamlining their explanation and making them easier to generalize and extrapolate.

show abstract

“…

Following the earlier formalism of the categorical representation learning [25] by the first two authors, we discuss the construction of the "RG-flow based categorifier". Borrowing ideas from theory of renormalization group flows (RG) in quantum field theory, holographic duality, and hyperbolic geometry, and mixing them with neural ODE's, we construct a new algorithmic natural language processing (NLP) architecture, called the RG-flow categorifier or for short the RG categorifier, which is capable of data classification and generation in all layers.

…”

mentioning

confidence: 99%

Categorical Representation Learning and RG flow operators for algorithmic classifiers

Sheshmani¹,

You²,

Fu³

et al. 2022

Preprint

Self Cite

0

View full text Add to dashboard Cite

Following the earlier formalism of the categorical representation learning [25] by the first two authors, we discuss the construction of the "RG-flow based categorifier". Borrowing ideas from theory of renormalization group flows (RG) in quantum field theory, holographic duality, and hyperbolic geometry, and mixing them with neural ODE's, we construct a new algorithmic natural language processing (NLP) architecture, called the RG-flow categorifier or for short the RG categorifier, which is capable of data classification and generation in all layers. We apply our algorithmic platform to biomedical data sets and show its performance in the field of sequenceto-function mapping. In particular we apply the RG categorifier to particular genomic sequences of flu viruses and show how our technology is capable of extracting the information from given genomic sequences, find their hidden symmetries and dominant features, classify them and use the trained data to make stochastic prediction of new plausible generated sequences associated with new set of viruses which could avoid the human immune system.

show abstract

Categorical representation learning: morphism is all you need

Abstract: We provide a construction for categorical representation learning and introduce the foundations of ‘categorifier’. The central theme in representation learning is the idea of everything to vector. Every object in a dataset S can be represented as a vector in … Show more

Cited by 5 publications

References 3 publications

Road Crash Injury Severity Prediction Using a Graph Neural Network Framework

Road Crash Injury Severity Prediction Using a Graph Neural Network Framework

Exploring explainable AI: category theory insights into machine learning algorithms

Categorical Representation Learning and RG flow operators for algorithmic classifiers

Contact Info

Product

Resources

About