Analysis of optimal thread pool size

Abstract

The success of e-commerce, messaging middleware, and other Internet-based applications depends in part on the ability of network servers to respond in a timely and reliable manner to simultaneous service requests. Multithreaded systems, due to their efficient use of system resources and the popularity of shared-memory multi-processor architectures, have become the server implementation of choice. However, creating and destroying a thread is far from free, requiring run-time memory allocation and deallocation. These overheads become especially onerous during periods of high load and can be a major factor behind system slowdowns. A thread-pool architecture addresses this problem by prespawning and then managing a pool of threads. Threads in the pool are reused, so that thread creation and destruction overheads are incurred only once per thread, and not once per request. However, efficient thread management for a given system load highly depends on the thread pool size, which is currently determined heuristically. In this paper, we characterize several system resource costs associated with thread pool size. If the thread pool is too large, and threads go unused, then processing and memory resources are wasted maintaining the thread pool. If the thread pool is too small, then additional threads must be created and destroyed on the fly to handle new requests. We analytically determine the optimal thread pool size to maximize the expected gain of using a thread.

A framework for identifying the linkability between Web servers for enhanced internet computing and E-commerce

“…Each analysis event maps to a thread that consumes memory to maintain its stack, utilizes the processor to run the lookup and the analysis processes, and incurs additional overhead resulting from thread context switches. For the computations, we suppose that the lookup process takes T l seconds to execute, the analysis process takes T a seconds, and each thread incurs a context switch of c a seconds, while thread creation and destruction overheads can be ignored as they are in the order of microseconds [32]. c) The query execution process responds to client (user or application) queries that aim to get a view of traffic flows involving certain Web servers.…”

Dynamic web worker pool management for highly parallel javascript web applications

“…Algorithms try to dynamically decide what is the optimal number of active threads to sustain required high performance. However, proposals that deal with native threads try to minimize the number of idle threads to reduce management overhead [12,13]. Our proposal is based on the fact that Web Workers run in a browser's virtual machine.…”

Application of queue and multi-thread in short message traceability system of broilers products quality

“…A high volume of service requests results in frequent memory allocation and deallocation, and becomes a major factor in performance bottleneck [8]. Under thread pool technology, maintaining threads in the pool incurs a run-time overhead for all tasks.…”