Hyperbolic Neural Networks++

Shimizu, Ryohei; Mukuta, Yusuke; Harada, Tatsuya

doi:10.48550/arxiv.2006.08210

Cited by 13 publications

(16 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several authors have recently adopted neural networks to learn representations in hyperbolic spaces in a wide variety of problems, e.g., classification, regression, detection, manifold learning, and high-dimensional distribution approximation. Hyperbolic neural networks have been developed to combine the representational power of hyperbolic geometry with the feature extraction capabilities of neural networks [ 12 , 13 ]. The main challenge in designing hyperbolic neural networks is performing hyperbolic arithmetic operations and deriving stable back-propagation formulas [ 12 , 13 , 15 ].…”

Section: Methodsmentioning

confidence: 99%

“…Hyperbolic neural networks have been developed to combine the representational power of hyperbolic geometry with the feature extraction capabilities of neural networks [ 12 , 13 ]. The main challenge in designing hyperbolic neural networks is performing hyperbolic arithmetic operations and deriving stable back-propagation formulas [ 12 , 13 , 15 ]. Learning in non-Euclidean spaces is quickly advancing with works such as constant curvature graph convolutional networks [ 18 , 19 ], hyperbolic graph neural networks [ 20 , 21 ], mixed-curvature variational autoencoders [ 22 ], and hyperbolic attention networks [ 23 ] to name a few.…”

Section: Methodsmentioning

confidence: 99%

“…These realizations have motivated machine learning researchers to propose methods to represent pairwise measurement extracted from a tree in hyperbolic spaces [ 9 , 10 , 11 ]. For example, hyperbolic neural networks seek to combine the power of hyperbolic geometry with the feature learning capabilities of neural networks [ 12 , 13 ]. A major goal of the present work is to take advantage of these recent developments for phylogenetic placement.…”

Section: Introductionmentioning

confidence: 99%

“…Another major challenge in using hyperbolic geometry is performing arithmetic operations using hyperbolic addition and multiplication. Various authors have approached these issues in different ways [ 12 , 13 , 15 ]. The present work wrestles with many of the same difficulties; for a detailed discussion, refer to Section 2.4.3 .…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Learning Hyperbolic Embedding for Phylogenetic Tree Placement and Updates

Jiang

Tabaghi

Mirarab

2022

Biology

View full text Add to dashboard Cite

Phylogenetic placement, used widely in ecological analyses, seeks to add a new species to an existing tree. A deep learning approach was previously proposed to estimate the distance between query and backbone species by building a map from gene sequences to a high-dimensional space that preserves species tree distances. They then use a distance-based placement method to place the queries on that species tree. In this paper, we examine the appropriate geometry for faithfully representing tree distances while embedding gene sequences. Theory predicts that hyperbolic spaces should provide a drastic reduction in distance distortion compared to the conventional Euclidean space. Nevertheless, hyperbolic embedding imposes its own unique challenges related to arithmetic operations, exponentially-growing functions, and limited bit precision, and we address these challenges. Our results confirm that hyperbolic embeddings have substantially lower distance errors than Euclidean space. However, these better-estimated distances do not always lead to better phylogenetic placement. We then show that the deep learning framework can be used not just to place on a backbone tree but to update it to obtain a fully resolved tree. With our hyperbolic embedding framework, species trees can be updated remarkably accurately with only a handful of genes.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning Hyperbolic Embedding for Phylogenetic Tree Placement and Updates

Jiang

Tabaghi

Mirarab

2022

Biology

View full text Add to dashboard Cite

show abstract

“…Hyperbolic Neural Networks (HNNs) [7] rewrite multinomial logitstic regression (MLR), fully connected layers and Recurrent Neural Networks for hyperbolic embeddings via gyrovector space operations [30]. In a follow-up work, Hyperbolic Neural Networks++ [28] introduced hyperbolic convolutional layers. In [8], the authors proposed Hyperbolic attention networks [8] by extending attention operations to hyperbolic space in a manner similar to [7].…”

Section: Related Workmentioning

confidence: 99%

CO-SNE: Dimensionality Reduction and Visualization for Hyperbolic Data

Guo¹,

Guo²,

Yu³

2021

Preprint

View full text Add to dashboard Cite

Hyperbolic space can embed tree metric with little distortion, a desirable property for modeling hierarchical structures of real-world data and semantics. While highdimensional embeddings often lead to better representations, most hyperbolic models utilize low-dimensional embeddings, due to non-trivial optimization as well as the lack of a visualization for high-dimensional hyperbolic data.We propose CO-SNE, extending the Euclidean space visualization tool, t-SNE, to hyperbolic space. Like t-SNE, it converts distances between data points to joint probabilities and tries to minimize the Kullback-Leibler divergence between the joint probabilities of high-dimensional data X and low-dimensional embeddings Y . However, unlike Euclidean space, hyperbolic space is inhomogeneous: a volume could contain a lot more points at a location far from the origin. CO-SNE thus uses hyperbolic normal distributions for X and hyberbolic Cauchy instead of t-SNE's Student's t-distribution for Y , and it additionally attempts to preserve X's individual distances to the Origin in Y .We apply CO-SNE to high-dimensional hyperbolic biological data as well as unsupervisedly learned hyperbolic representations. Our results demonstrate that CO-SNE deflates high-dimensional hyperbolic data into a lowdimensional space without losing their hyperbolic characteristics, significantly outperforming popular visualization tools such as PCA, t-SNE, UMAP, and HoroPCA, the last of which is specifically designed for hyperbolic data.

show abstract

Adaptive frequency-based fully hyperbolic graph neural networks

Wei,

Ping,

Mei

2023

Pattern Anal Applic

View full text Add to dashboard Cite

Graph Convolutional Networks (GCNs) have attracted broad attention from industry and academia, for which GCNs have demonstrated powerful ability to model the irregular data, e.g., skeletal data and graph-structured data. The most existing effective model may be the fully hyperbolic graph neural network. However, it involves a large number of parameters, thus consuming considerable computing resources. In this paper, we propose a model based on adaptive frequency filter and corresponding optimizer in hyperbolic space. The adaptive frequency can learn the different frequency components of the embeddings of the nodes in graph, which adaptively adjust the beneficial signals of high-frequency and lowfrequency. And the optimizer is based on a subset of the orthogonal constraint, which is dedicated for the adaptive frequency with less parameters. Consequently, our model need only to optimize the less parameters in hyperbolic space and meanwhile prevent the distortion caused by conventional manifold GCN. Experimental results show that our method achieves substantial improvements and outperforms the state-of-the-art performance in terms of node classification.

show abstract

Hyperbolic Neural Networks++

Cited by 13 publications

References 19 publications

Learning Hyperbolic Embedding for Phylogenetic Tree Placement and Updates

Learning Hyperbolic Embedding for Phylogenetic Tree Placement and Updates

CO-SNE: Dimensionality Reduction and Visualization for Hyperbolic Data

Adaptive frequency-based fully hyperbolic graph neural networks

Contact Info

Product

Resources

About