Performance analysis of SIIT implementations: Testing and improving the methodology

Kovács

Shima

2020

IJATES

Self Cite

In this paper, we investigate three potential issues of the benchmarking measurement procedures defined in RFC 2544 and also used in RFC 5180 and RFC 8219. One of them is the lack of proper timeout usage. We use a Linux box, which can selectively delay a specified ratio of the packets. Using carefully selected parameters based on our preliminary measurements, we demonstrate that the experienced speed of the HTTP download is much less, than what could have been expected on the basis of the throughput results of the RFC 2544 tests. The other critical issue is the strict, absolutely zero loss criterion. We use a Linux box, which drops a specified small ratio of the packets. Whereas the RFC 2544 throughput results tend to be zero, the experienced speed of the HTTP download is quite good. The third problem is the lack of requirement for statistically relevant number of tests in the RFC 2544 benchmarking procedures. We demonstrate its severity with the latency benchmarking procedure defined in RFC 2544 and kept unchanged in RFC 5180 but redefined in RFC 8219.

Section: Latency Measurementsmentioning

confidence: 99%

Section: Latency Measurementsmentioning

confidence: 99%

See 1 more Smart Citation

Gaming with the Throughput and the Latency Benchmarking Measurement Procedures of RFC 2544

Kovács

Shima

2020

IJATES

Self Cite

“…The same implementations were tested, but Jool was the most current version, that is 4.0.1. Our results will be published in [29]. Unfortunately, the test program had several issues, which we could temporarily fix, but we plan to redesign and re-implement it in C++.…”

Section: Further Work and Our Future Plansmentioning

confidence: 99%

Benchmarking stateless NAT64 implementations with a standard tester

2020

Telecommun Syst

RFC 5180, the IPv6 update of RFC 2544, declared IPv6 transition technologies out of its scope. RFC 8219 defined a benchmarking methodology for IPv6 transition technologies including stateless NAT64 (more properly called SIIT) in the category of single translation solutions. Whereas several research papers have dealt with the performance of different stateful NAT64 implementations, none of them used RFC 8219 compliant measurements or addressed stateless NAT64 implementations. In this paper, we show, how stateless NAT64 implementations can be benchmarked carrying out the most important tests recommended by RFC 8219 without a special purpose NAT64 Tester, using simply an RFC 2544/RFC 5180 compliant legacy Tester. We carry out benchmarking measurements to examine the performance of three free software NAT64 implementations, namely: Jool, TAYGA and map646. All the details of our measurements are disclosed and their results are presented in the paper.

“…On the one hand, the usage of fixed port numbers (together with fixed IP addresses) allows the reuse of the very same test frames, which can be a performance advantage for software testers. However, on the other hand, our SIIT benchmarking experience showed that the usage of fixed test frames resulted in a situation, were only two CPU cores were used 1 (one core for each direction) from the several cores of a the two CPUs of the computer used as the DUT (currently: SIIT gateway) [11]. We believe that the results of such measurements do not reflect the real life performance of a multi-core DUT well enough, because a high number of different IP addresses and different port numbers occur in a real life traffic, thus the interrupts are hashed more or less equally to all CPU cores.…”

Section: Introductionmentioning

confidence: 99%

Adding RFC 4814 Random Port Feature to Siitperf: Design, Implementation and Performance Estimation

2020

IJATES

Self Cite

Siitperf is the World’s first free software RFC 8219 compliant SIIT (also called stateless NAT64) tester written in C++ using DPDK, which is also suitable for benchmarking IPv4 / IPv6 network interconnect devices in RFC 2544 / RFC 5180 compliant ways. Originally, siitperf followed RFC 2544 Appendix C.2.6.4 test frame format resulting in “hard coded” source and destination UDP port numbers. RFC 4814 Section 4.5 recommended random, uniformly distributed source and destination port numbers, which can make a very significant difference, when the DUT (Device Under Test) has multiple CPU cores, what is very common today. Therefore, adding this feature to siitperf is essential to be able to produce meaningful benchmarking results. In this paper, we disclose the design, implementation and performance estimation of this extension of siitperf.