Graph convolutional networks in language and vision: A survey

Ren, Haotian; Lu, Wei; Xiao, Yun; Chang, Xiaojun; Wang, Xuanhong; Dong, Zhiqiang; Fang, Dingyi

doi:10.1016/j.knosys.2022.109250

Cited by 25 publications

(5 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, generating the graphs can be computationally demanding. This is particularly true when dealing with high-density event streams, resulting in a large number of vertices and edges [54]. Consequently, it is often necessary to sample a subset of events from the batch to reduce storage and computational costs [20], [53].…”

Section: ) Graph-based Representationsmentioning

confidence: 99%

CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision

El Shair,

Hassani,

Rawashdeh

2023

IEEE Access

View full text Add to dashboard Cite

Event-based vision is a novel perception modality that offers several advantages, such as high dynamic range and robustness to motion blur. In order to process events in batches and utilize modern computer vision deep-learning architectures, an intermediate representation is required. Nevertheless, constructing an effective batch representation is non-trivial. In this paper, we propose a novel representation for event-based vision, called the compact spatio-temporal representation (CSTR). The CSTR encodes an event batch's spatial, temporal, and polarity information in a 3-channel image-like format. It achieves this by calculating the mean of the events' timestamps in combination with the event count at each spatial position in the frame. This representation shows robustness to motion-overlapping, high event density, and varying event-batch durations. Due to its compact 3-channel form, the CSTR is directly compatible with modern computer vision architectures, serving as an excellent choice for deploying event-based solutions. In addition, we complement the CSTR with an augmentation framework that introduces randomized training variations to the spatial, temporal, and polarity characteristics of event data. Experimentation over different object and action recognition datasets shows that the CSTR outperforms other representations of similar complexity under a consistent baseline. Further, the CSTR is made more robust and significantly benefits from the proposed augmentation framework, considerably addressing the sparseness in event-based datasets.

show abstract

Section: ) Graph-based Representationsmentioning

confidence: 99%

CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision

El Shair,

Hassani,

Rawashdeh

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Ahmad et al [54] examined the use of Graph Convolutional Neural Networks (GCNs) in human action recognition and proposed a taxonomy of studies in this area. Other studies have provided detailed insights into the use of GNNs in various applications such as the Internet of Things (IoT) [55], network science [56], and language processing [57]. There are also general information on accelerators and efficient GNNs [29,58,59].…”

Section: Introductionmentioning

confidence: 99%

A Survey of Computationally Efficient Graph Neural Networks for Reconfigurable Systems

Kose,

Nunez-Yanez,

Piechocki

et al. 2024

Preprint

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) are powerful models capable of managing intricate connections in non-Euclidean data, such as social networks, physical systems, chemical structures, and communication networks. Despite their effectiveness, the large-scale and complex nature of graph data demands substantial computational resources and high performance during both training and inference stages, presenting significant challenges, particularly in the context of embedded systems. Recent studies on GNNs have investigated both software and hardware solutions to enhance computational efficiency. Earlier studies on Deep Neural Networks (DNNs) have indicated that methods like reconfigurable hardware and quantization are beneficial in addressing these issues. Unlike DNN research, studies on efficient computational methods for GNNs are less developed and require more exploration. This survey reviews the latest developments in quantization and FPGA-based acceleration for GNNs, showcasing the capabilities of reconfigurable systems (often FPGAs) to offer customized solutions in environments marked by significant sparsity and the necessity for dynamic load management. It also emphasizes the role of quantization in reducing both computational and memory demands through the use of fixed-point arithmetic and streamlined vector formats. This paper concentrates on low-power, resource-limited devices over general hardware accelerators and reviews research applicable to embedded systems. Additionally, it provides a detailed discussion of potential research gaps, foundational knowledge, obstacles, and prospective future directions.

show abstract

“…Graph Convolutional Networks (GCNs) [1] introduced a convolution component designed for graphs, where vertices are allowed to have a varying number of neighbors unlike fixed grids. GCNs were used in several application such as sentiment analysis [2] , computer vision tasks [3] and ranking gas adsorption properties [4] .…”

Section: Introductionmentioning

confidence: 99%

Random projection forest initialization for graph convolutional networks

Alshammari,

Stavrakakis,

Ahmed

et al. 2023

MethodsX

View full text Add to dashboard Cite

Graph convolutional networks in language and vision: A survey

Cited by 25 publications

References 66 publications

CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision

CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision

A Survey of Computationally Efficient Graph Neural Networks for Reconfigurable Systems

Random projection forest initialization for graph convolutional networks

Contact Info

Product

Resources

About