Predicting the Characteristics of High-Speed Serial Links Based on a Deep Neural Network (DNN)—Transformer Cascaded Model

Abstract

The design level of channel physical characteristics has a crucial influence on the transmission quality of high-speed serial links. However, channel design requires a complex simulation and verification process. In this paper, a cascade neural network model constructed of a Deep Neural Network (DNN) and a Transformer is proposed. This model takes physical features as inputs and imports a Single-Bit Response (SBR) as a connection, which is enhanced through predicting frequency characteristics and equalizer parameters. At the same time, signal integrity (SI) analysis and link optimization are achieved by predicting eye diagrams and channel operating margins (COMs). Additionally, Bayesian optimization based on the Gaussian process (GP) is employed for hyperparameter optimization (HPO). The results show that the DNN–Transformer cascaded model achieves high-precision predictions of multiple metrics in performance prediction and optimization, and the maximum relative error of the test-set results is less than 2% under the equalizer architecture of a 3-taps TX FFE, an RX CTLE with dual DC gain, and a 12-taps RX DFE, which is more powerful than other deep learning models in terms of prediction ability.