Topological limits to the parallel processing capability of network architectures

Petri, Giovanni; Musslick, Sebastian; Dey, Biswadip; Özcimder, Kayhan; Turner, D. T.; Ahmed, Nesreen K.; Willke, Theodeore L.; Cohen, Jonathan D.

doi:10.1038/s41567-021-01170-x

Cited by 24 publications

(28 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multipleresource theories [41,42,44,45] became increasingly successful in explaining multitasking phenomena in laboratory tasks, such as higher dual-task interference for structurally overlapping tasks [46,47], and in real-world scenarios, such as the effects of phone dialing on speed control while driving [48,49]. In addition, these theories are supported by recent numerical and analytical work, suggesting that even modest amounts of resource sharing between tasks can be sufficient to drastically limit the multitasking capacity of a neural system [50][51][52] and that this effect scales with the number of processing steps (layers) in the network [53]. Thus, even small amounts of representation sharing are sufficient to induce constraints on multitasking that may invite misinterpretation as a central bottleneck [54].…”

Section: Open Accessmentioning

confidence: 99%

“…can be recombined to form another (interfering) task [51,52]. However, despite growing empirical and quantitative support, multiple-resource theories lack a principled explanation for why a neural system, such as the human brain, would rely on shared resources between tasks at all, given the constraints on multitasking that this imposes.…”

Section: Open Accessmentioning

confidence: 99%

“…To our knowledge, there has not yet been a direct empirical test of this prediction, although the effects of structural overlap between tasks in both dual-task and task switching paradigms suggest such a relationship [46,[128][129][130]. This prediction extends from structural overlap on which most previous instances of multiple resource theory have focused [131], to the case of functional dependence addressed by crosstalk models [27,47,51,52,132]. Even if two tasks do not directly share representations (e.g., color naming and word pointing in Figure 2C), they may still interfere with one another by means of another process (e.g., word reading).…”

Section: Concluding Remarks and Future Directionsmentioning

confidence: 99%

See 2 more Smart Citations

Rationalizing constraints on the capacity for cognitive control

Musslick

Cohen

2021

Trends in Cognitive Sciences

Self Cite

143

View full text Add to dashboard Cite

Humans are remarkably limited in: (i) how many control-dependent tasks they can execute simultaneously, and (ii) how intensely they can focus on a single task. These limitations are universal assumptions of most theories of cognition. Yet, a rationale for why humans are subject to these constraints remains elusive. This feature review draws on recent insights from psychology, neuroscience, and machine learning, to suggest that constraints on cognitive control may result from a rational adaptation to fundamental, computational dilemmas in neural architectures. The reviewed literature implies that limitations in multitasking may result from a trade-off between learning efficacy and processing efficiency and that limitations in the intensity of commitment to a single task may reflect a trade-off between cognitive stability and flexibility. The role of capacity constraints in human cognitionOne of the most remarkable features of human cognition is the ability to rapidly adapt behavior in a changing world. This is often attributed to the capacity for cognitive control: the ability to flexibly direct behavior in pursuit of a goal (Box 1). Cognitive control is engaged by all of the higher mental faculties that distinguish humans from other species, including reasoning, problem solving, planning, and the use of symbolic language [1]. Yet, humans are strikingly limited in how many control-demanding tasks (see Glossary) they can perform simultaneously (e.g., reading a document while listening to a friend) or how intensely they can focus on a single task (e.g., parsing a mathematical equation in a noisy environment). The significance of these limitations is not only apparent in daily life. They are also a fundamental premise of general theories of human cognition (e.g. [2-7],). These theories posit that the exertion of cognitive control is associated with a cost, and that humans consider this cost when making decisions about how to allocate control [7][8][9]. The notion of a cost and concomitant constraints on control, can help integrate a wide range of empirical findings concerning the allocation of mental effort [10][11][12][13], the selection between cognitive heuristics [5], planning [14,15], or cognitive impairments in depression [16]. Yet, none of these theories provides an explanation for why control-dependent processing would be subject to these limitations in the first place.Here, we review two fundamental, computational dilemmas that arise in neural systems and suggest that these provide a rational account of constraints on cognitive control. First, we review empirical and computational evidence suggesting a trade-off between the rapid acquisition of novel tasks (learning efficacy), that is promoted by sharing representations across tasks, on the one hand; and multitasking capability (processing efficiency) that is achieved by separating representations and dedicating them to individual tasks, on the other hand. The work reviewed suggests that limitations in the ability to simultaneously execute multiple tasks may...

show abstract

Section: Open Accessmentioning

confidence: 99%

Section: Open Accessmentioning

confidence: 99%

Section: Concluding Remarks and Future Directionsmentioning

confidence: 99%

See 1 more Smart Citation

Rationalizing constraints on the capacity for cognitive control

Musslick

Cohen

2021

Trends in Cognitive Sciences

Self Cite

143

View full text Add to dashboard Cite

show abstract

“…Moreover, in support of the link between multitasking and performance generalisability, have shown that neural networks that allow overlap between task representations show both decrements in multitasking performance and more rapid acquisition of new tasks. Importantly, representational sharing rapidly attenuates multitasking performance independent of network size (Petri et al, 2021), perhaps accounting for how a system as complex as the human brain can still show striking multitasking limitations.…”

Section: Introductionmentioning

confidence: 99%

When learning hurts control: inducing cross-task-covariance causes multitasking costs

Garner¹,

Butler²,

Jones³

et al. 2021

Preprint

View full text Add to dashboard Cite

Performing two tasks concurrently typically leads to performance costs. Historically, multitasking costs have been assumed to reflect fundamental constraints of cognitive architectures. A new perspective proposes that multitasking costs reflect information sharing between constituent tasks; shared information gains representational efficiency, at the expense of multitasking capability. We test this theory by determining whether increasing cross-task information harms multitasking. 48 participants performed multitasks where they mapped keypresses to four shapes. In a subsequent statistical learning task, these shapes then formed pairs that were predictive or non-predictive of an upcoming target judgement. When participants again responded to these shapes in the multitasking context, performance was poorer when the shape pair had been predictive of target outcomes in the learning phase, relative to non-predictive. Thus, associating common information to shape pairings transferred to negatively impact multitasking performance, providing the first causal evidence for the shared representational account of multitasking performance.

show abstract

“…Here we explore the topology attacks and injection attacks for the deep networks. Motivated by the topological limits to the parallel processing capability of the multitask networks 16 . The deep networks can give play to the maximum work capability by removing the nodes, which is described as dropout 17 .…”

Section: Introductionmentioning

confidence: 99%

Working capacity of deep networks with topology attacks

Zhang

Chen

Ali

2021

Preprint

View full text Add to dashboard Cite

Deep learning plays an important role in the development of artificial intelligence (AI) technology. The security of deep networks has become the crucial thing to be considered. When the deep learning algorithms are implemented in the hardware platform, the interference for topology structure will appear because of cyber-attacks. We analyze the working capacity of acyclic deep networks under the topology attacks and injection attacks. Considering the topology structure of the deep network, the maximum working capacity is studied under the topology attacks and injection attacks. Furthermore, the robustness of the random networks is researched and the structural robustness index (SRI) is proposed to measure the toleration for the topology attacks. This work supplies some suggestions for building a robust deep network and improving the endogenous safety and security (ESS) of the deep networks.

show abstract

Topological limits to the parallel processing capability of network architectures

Cited by 24 publications

References 27 publications

Rationalizing constraints on the capacity for cognitive control

Rationalizing constraints on the capacity for cognitive control

When learning hurts control: inducing cross-task-covariance causes multitasking costs

Working capacity of deep networks with topology attacks

Contact Info

Product

Resources

About