Delay and Bypass: Ready and Criticality Aware Instruction Scheduling in Out-of-Order Processors

Alipour, Mehdi; Black-Schaffer, David; Kumar, Rakesh

doi:10.1109/hpca47549.2020.00042

Cited by 10 publications

(14 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Long-term parking [33] saves power in an OoO core by allocating back-end resources for critical instructions while buffering non-critical instructions in the frontend. More recently, Alipour et al [2] leverage instruction criticality and readiness to bypass the outof-order back-end. Instructions that do not benefit from out-of-order scheduling and instructions that do not suffer from being delayed are sent to an in-order FIFO queue.…”

Section: Related Workmentioning

confidence: 99%

The Forward Slice Core: A High-Performance, Yet Low-Complexity Microarchitecture

Lakshminarasimhan

Naithani

Feliu

et al. 2022

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

Superscalar out-of-order cores deliver high performance at the cost of increased complexity and power budget. In-order cores, in contrast, are less complex and have a smaller power budget, but offer low performance. A processor architecture should ideally provide high performance in a power- and cost-efficient manner. Recently proposed slice-out-of-order (sOoO) cores identify backward slices of memory operations which they execute out-of-order with respect to the rest of the dynamic instruction stream for increased instruction-level and memory-hierarchy parallelism. Unfortunately, constructing backward slices is imprecise and hardware-inefficient, leaving performance on the table. In this article, we propose Forward Slice Core (FSC ), a novel core microarchitecture that builds on a stall-on-use in-order core and extracts more instruction-level and memory-hierarchy parallelism than slice-out-of-order cores. FSC does so by identifying and steering forward slices (rather than backward slices) to dedicated in-order FIFO queues. Moreover, FSC puts load-consumers that depend on L1 D-cache misses on the side to enable younger independent load-consumers to execute faster. Finally, FSC eliminates the need for dynamic memory disambiguation by replicating store-address instructions across queues. Considering 3-wide pipeline configurations, we find that FSC improves performance by 27.1%, 21.1%, and 14.6% on average compared to Freeway, the state-of-the-art sOoO core, across SPEC CPU2017, GAP, and DaCapo, respectively, while at the same time incurring reduced hardware complexity. Compared to an OoO core, FSC reduces power consumption by 61.3% and chip area by 47%, providing a microarchitecture with high performance at low complexity.

show abstract

Section: Related Workmentioning

confidence: 99%

The Forward Slice Core: A High-Performance, Yet Low-Complexity Microarchitecture

Lakshminarasimhan

Naithani

Feliu

et al. 2022

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

show abstract

“…Long-term parking [16] saves power in an OoO core by allocating back-end resources for critical instructions while buffering non-critical instructions in the front-end. More recently, Alipour et al [1] leverage instruction criticality and readiness to bypass the out-of-order back-end. Instructions that do not benefit from out-of-order scheduling and instructions that do not suffer from being delayed are sent to an in-order FIFO queue.…”

Section: Related Workmentioning

confidence: 99%

“…The sOoO cores are restricted out-of-order machines that add modest hardware overhead upon a stall-on-use in-order core to improve instruction-level parallelism (ILP) as well as memory-hierarchy parallelism (MHP). 1 Load Slice Core (LSC) [5] was the first work to propose an sOoO core; Freeway [10] builds upon the LSC proposal and exposes more MHP than LSC by adding one more in-order queue for uncovering additional independent loads. LSC and Freeway identify the address-generating instructions (AGIs) of loads and stores in an iterative manner using a hardware mechanism called Iterative Backward Dependence Analysis (IBDA).…”

mentioning

confidence: 99%

The Forward Slice Core Microarchitecture

Lakshminarasimhan

Naithani

Feliu

et al. 2020

Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques

View full text Add to dashboard Cite

Superscalar out-of-order cores deliver high performance at the cost of increased complexity and power budget. In-order cores, in contrast, are less complex and have a smaller power budget, but offer low performance. A processor architecture should ideally provide high performance in a power-and cost-efficient manner. Recently proposed slice-out-of-order (sOoO) cores identify backward slices of memory operations which they execute out-of-order with respect to the rest of the dynamic instruction stream for increased instruction-level and memory-hierarchy parallelism. Unfortunately, constructing backward slices is imprecise and hardware-inefficient, leaving performance on the table. In this paper, we propose Forward Slice Core (FSC), a novel core microarchitecture that builds on a stall-on-use in-order core and extracts more instruction-level and memory-hierarchy parallelism than slice-out-of-order cores. FSC does so by identifying and steering forward slices (rather than backward slices) to dedicated inorder FIFO queues. Moreover, FSC puts load-consumers that depend on L1 D-cache misses on the side to enable younger independent load-consumers to execute faster. Finally, FSC eliminates the need for dynamic memory disambiguation by replicating store-address instructions across queues. FSC improves performance by 9.7% on average compared to Freeway, the state-of-the-art sOoO core, across the SPEC CPU2017 benchmarks, while incurring reduced hardware complexity and a similar power budget.

show abstract

“…Latency-tolerating techniques such as out-of-order (OOO) execution [116] continue to execute independent instructions behind long-latency loads in the sequential instruction stream, avoiding pipeline stalls. The opportunities for reordering instructions have been further extended by systems that consider instruction criticality [3,20,66,81,90,91,102] and increase the number of instructions that can be reordered with delinquent loads.…”

mentioning

confidence: 99%

“…Continuous runahead engines [48] also introduce additional compute cores for executing (redundant) instructions to generate prefetches. OOO processors have exploited criticality mainly for improving energy efficiency [3,20,66,81,90,91,102] and for optimizing the cache hierarchy [10,87,115] but not for prefetching. These techniques also introduce complex hardware and storage requirements for classifying critical instructions at runtime.…”

mentioning

confidence: 99%

CRISP: critical slice prefetching

Litz

Ayers

Ranganathan

2022

Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

View full text Add to dashboard Cite

The high access latency of DRAM continues to be a performance challenge for contemporary microprocessor systems. Prefetching is a well-established technique to address this problem, however, existing implemented designs fail to provide any performance benefits in the presence of irregular memory access patterns. The hardware complexity of prior techniques that can predict irregular memory accesses such as runahead execution has proven untenable for implementation in real hardware. We propose a lightweight mechanism to hide the high latency of irregular memory access patterns by leveraging criticality-based scheduling. In particular, our technique executes delinquent loads and their load slices as early as possible, hiding a significant fraction of their latency. Furthermore, we observe that the latency induced by branch mispredictions and other high latency instructions can be hidden with a similar approach. Our proposal only requires minimal hardware modifications by performing memory access classification, load and branch slice extraction, as well as priority analysis exclusively in software. As a result, our technique is feasible to implement, introducing only a simple new instruction prefix while requiring minimal modifications of the instruction scheduler. Our technique increases the IPC of memory-latency-bound applications by up to 38% and by 8.4% on average. CCS CONCEPTS• Computer systems organization → Superscalar architectures.

show abstract

Delay and Bypass: Ready and Criticality Aware Instruction Scheduling in Out-of-Order Processors

Cited by 10 publications

References 37 publications

The Forward Slice Core: A High-Performance, Yet Low-Complexity Microarchitecture

The Forward Slice Core: A High-Performance, Yet Low-Complexity Microarchitecture

The Forward Slice Core Microarchitecture

CRISP: critical slice prefetching

Contact Info

Product

Resources

About