Approaches of enhancing interoperations among high performance computing and big data analytics via augmentation

Pathak, Ajeet Ram; Pandey, Manjusha; Rautaray, Siddharth Swarup

doi:10.1007/s10586-019-02960-y

Cited by 9 publications

(2 citation statements)

References 124 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…erefore, it is a typical combinatorial optimization problem, which aims to study discrete space through certain mathematical methods. e solution of combinatorial optimization problem is mainly divided into two parts: mathematical model establishment and optimal solution search process [16]. e intelligent optimization algorithm is the sum of the theoretical methods to find the optimal solution of the problem under the condition of feasible solutions and constraints through a certain optimization process based on a certain search mechanism.…”

Section: Multicycle Order Allocation Distribution Designmentioning

confidence: 99%

Dynamic Scheduling and Optimal Control of Coordination Supply Chain Based on Automatic Allocation of Orders for Multicycle Integrators

2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Selecting suppliers and allocating orders scientifically and rationally not only have important theoretical value but also have extremely important practical significance to enterprises. This study proposes an improved multicycle integration method that can effectively determine the optimal number of suppliers under the premise of considering the risk of supply interruption and transaction costs that may be caused by various uncertain factors. The supply interruption risk is divided into two categories: “common risk event” and “individual risk event.” On the basis of balancing the transaction cost of the company’s choice of suppliers and the financial loss of supply interruption, quantitative analysis methods are used to determine the optimal number of suppliers for the company. The problem is analyzed, the corresponding model is given, and the influence of each parameter on the optimal number of suppliers is analyzed. From the perspective of stochastic uncertainty in both customer demand and supplier supply capacity, this study analyzes the optimization problem of enterprise purchase quantity allocation under the volume discount environment. The relaxation factor is somewhat adaptive because the particle swarm is continuously updated. Aiming at ordering cost, transaction cost, transportation cost, order delay quantity, and product quality, considering random demand constraints, supplier supply capacity constraints, and multisupplier multiproduct situations, we establish supplier selection in an environment of uncertain demand and supply capacity and dynamic relaxation approximation algorithm for order allocation. The research results show that the dynamic relaxation approximation algorithm can not only determine the optimal supplier but also determine the optimal procurement allocation for each supplier.

show abstract

Section: Multicycle Order Allocation Distribution Designmentioning

confidence: 99%

Dynamic Scheduling and Optimal Control of Coordination Supply Chain Based on Automatic Allocation of Orders for Multicycle Integrators

2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…While these modifications enhance QP scalability, the high cost and potential impact on NIC reliability make them impractical for deployment in hyper scale AI infrastructure. Additionally, congestion control algorithms like DCQCN and TIMELY, used in these studies, cannot completely prevent collisions of multiple large flows at specific nodes (usually refers to servers or switches) in extremely large-scale AI training networks [6,9,11,18], leading to congestion in network traffic at those points.…”

Section: Introductionmentioning

confidence: 99%

An Optimized RDMA QP Communication Mechanism for Hyperscale AI Infrastructure

Wang,

Lin,

Zhang

et al. 2024

Preprint

View full text Add to dashboard Cite

The current artificial intelligence (AI) infrastructure widely employs remote direct memory access (RDMA) protocol for high-performance communication in networks, utilizing Reliable Connection (RC)-based Queue Pairs (QP) to ensure end-to-end correct and ordered data transmission. However, as the scale of AI infrastructure continues to expand, this RC-based QP communication mechanism faces deficiencies in scalability and is prone to congestion, resulting in degraded network transfer performance. In this paper, we propose an optimized RDMA QP communication mechanism to address scalability and congestion issues in hyper-scale AI infrastructure networks. Firstly, we replace RC-based QPs with Reliable Datagram (RD)-based QPs and propose a new reliable mechanism to address scalability problems, eliminating the need for repetitive QP establishment by AI processes during external communication. Additionally, to mitigate congestion caused by a single path, we implement multipath data transmission by introducing a new unordered reception method in the network software stack. Through experiments and simulation tests, the optimized RDMA QP communication in large-scale AI infrastructure exhibits exceptional scalability and significantly reduces the occurrence of congestion, resulting in an overall network performance improvement of over 15%.

show abstract

Avoiding Resource Wastage

Sampaio

Barbosa

2023

High Performance Computing in Clouds

View full text Add to dashboard Cite

Approaches of enhancing interoperations among high performance computing and big data analytics via augmentation

Cited by 9 publications

References 124 publications

Dynamic Scheduling and Optimal Control of Coordination Supply Chain Based on Automatic Allocation of Orders for Multicycle Integrators

Dynamic Scheduling and Optimal Control of Coordination Supply Chain Based on Automatic Allocation of Orders for Multicycle Integrators

An Optimized RDMA QP Communication Mechanism for Hyperscale AI Infrastructure

Avoiding Resource Wastage

Contact Info

Product

Resources

About