Towards a General Attention Mechanism for Embedded Intelligent Systems

Helgason, Helgi Páll; Þórisson, Kristinn R.; Garrett, Deon; Nivel, Eric

doi:10.5963/ijcsai0401001

Cited by 3 publications

(3 citation statements)

References 14 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the Auto-catalytic Endogenous Reflective Architecture (AERA) attention is implemented as system-permeating control of computational/cognitive resources at very fine-grain levels of processing, bounded by goals at one end and the current situation at the other (cf. Helgason et al, 2013 ; Nivel et al, 2015 ). Studies on multitasking in humans have shown that a degree of parallelism among multiple tasks is more likely if the tasks involve different data modalities, such as linguistic and tactile.…”

Section: Related Work and Attention's Role In Cognitionmentioning

confidence: 99%

Neurosymbolic Systems of Perception and Cognition: The Role of Attention

et al. 2022

View full text Add to dashboard Cite

A cognitive architecture aimed at cumulative learning must provide the necessary information and control structures to allow agents to learn incrementally and autonomously from their experience. This involves managing an agent's goals as well as continuously relating sensory information to these in its perception-cognition information processing stack. The more varied the environment of a learning agent is, the more general and flexible must be these mechanisms to handle a wider variety of relevant patterns, tasks, and goal structures. While many researchers agree that information at different levels of abstraction likely differs in its makeup and structure and processing mechanisms, agreement on the particulars of such differences is not generally shared in the research community. A dual processing architecture (often referred to as System-1 and System-2) has been proposed as a model of cognitive processing, and they are often considered as responsible for low- and high-level information, respectively. We posit that cognition is not binary in this way and that knowledge at any level of abstraction involves what we refer to as neurosymbolic information, meaning that data at both high and low levels must contain both symbolic and subsymbolic information. Further, we argue that the main differentiating factor between the processing of high and low levels of data abstraction can be largely attributed to the nature of the involved attention mechanisms. We describe the key arguments behind this view and review relevant evidence from the literature.

show abstract

Section: Related Work and Attention's Role In Cognitionmentioning

confidence: 99%

Neurosymbolic Systems of Perception and Cognition: The Role of Attention

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Human attention is a built-in mechanism for deciding how to apply their brainpower from moment to moment, (e.g., decide where to see in saliency visual object detection [27]). Attention mechanism is a reasonably well studied subject within the field of cognitive psychology and is known to be a key feature of human artificial intelligence [28]. Nowadays, attention-based deep learning methods are active especially in dealing problems concerning sequence prediction or control, including object detection, natural language processing, and deep reinforcement learning.…”

Section: Attention Mechanism Applied To Deep Learningmentioning

confidence: 99%

Cognition-Based Deep Learning: Progresses and Perspectives

Yi¹,

Chen²,

Chen³

et al. 2018

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

The human brain is composed of multiple modular subsystems, with a unique way interacting among. These subsystems have their own unique characteristics and interact to support cognitive functions such as memory, attention and cognitive control. Nowadays, deep learning methods based on the above-mentioned functions accompanied with knowledge are widely used to design more dynamic, robust and powerful systems. We first review and summarize the progresses of cognition-based deep neural networks, and how cognitive mechanisms can be applied to more brain-like neural networks. Then we propose a general framework for the design of cognition-based deep learning system. Although great efforts have been made in this field, cognition-based deep learning is still in its early age. We put forward the potential directions towards this field, such as associative memory in deep learning, interpretable network with cognitive mechanisms, and deep reinforcement learning based on cognitive science.

show abstract

“…3) Model Spaces: The necessity to maintain highdimensional representations of the state space X poses a major challenge for current approaches to learning [51], [52], [22] and general problem solving [53], [54]. The method closest to ours in its formalism seems to be that of [29] and even lends itself to learning by a connectionist network [55] -but still requires an exponentially large representation for planning purposes.…”

Section: Overview and Related Literaturementioning

confidence: 99%

Universal Memory Architectures for Autonomous Machines

Guralnik,

Koditschek

2015

Preprint

View full text Add to dashboard Cite

We propose a self-organizing memory architecture for perceptual experience capable of supporting autonomous learning and goal-directed problem solving in the absence of any prior information about the agent's environment. The architecture is simple enough to ensure (1) a quadratic bound (in the number of available sensors) on space requirements, and (2) a quadratic bound on the time-complexity of the update-execute cycle. At the same time, it is sufficiently complex to provide the agent with an internal representation which is (3) minimal among all representations of its class which account for every sensory equivalence class subject to the agent's belief state; (4) capable, in principle, of recovering the homotopy type of the system's state space; (5) learnable with arbitrary precision through a random application of the available actions.The provable properties of an effectively trained memory structure exploit a duality between weak poc sets -a symbolic (discrete) representation of subset nesting relations -and nonpositively curved cubical complexes, whose rich convexity theory underlie the planning cycle of the proposed architecture.

show abstract

Towards a General Attention Mechanism for Embedded Intelligent Systems

Cited by 3 publications

References 14 publications

Neurosymbolic Systems of Perception and Cognition: The Role of Attention

Neurosymbolic Systems of Perception and Cognition: The Role of Attention

Cognition-Based Deep Learning: Progresses and Perspectives

Universal Memory Architectures for Autonomous Machines

Contact Info

Product

Resources

About