A Novel Evaluation Metric for Deep Learning-Based Side Channel Analysis and Its Extended Application to Imbalanced Data

Abstract

Since Kocher (CRYPTO’96) proposed timing attack, side channel analysis (SCA) has shown great potential to break cryptosystems via physical leakage. Recently, deep learning techniques are widely used in SCA and show equivalent and even better performance compared to traditional methods. However, it remains unknown why and when deep learning techniques are effective and efficient for SCA. Masure et al. (IACR TCHES 2020(1):348–375) illustrated that deep learning paradigm is suitable for evaluating implementations against SCA from a worst-case scenario point of view, yet their work is limited to balanced data and a specific loss function. Besides, deep learning metrics are not consistent with side channel metrics. In most cases, they are deceptive in foreseeing the feasibility and complexity of mounting a successful attack, especially for imbalanced data. To mitigate the gap between deep learning metrics and side channel metrics, we propose a novel Cross Entropy Ratio (CER) metric to evaluate the performance of deep learning models for SCA. CER is closely related to traditional side channel metrics Guessing Entropy (GE) and Success Rate (SR) and fits to deep learning scenario. Besides, we show that it works stably while deep learning metrics such as accuracy becomes rather unreliable when the training data tends to be imbalanced. However, estimating CER can be done as easy as natural metrics in deep learning algorithms with low computational complexity. Furthermore, we adapt CER metric to a new kind of loss function, namely CER loss function, designed specifically for deep learning in side channel scenario. In this way, we link directly the SCA objective to deep learning optimization. Our experiments on several datasets show that, for SCA with imbalanced data, CER loss function outperforms Cross Entropy loss function in various conditions.

Imbalanced Data Problems in Deep Learning-Based Side-Channel Attacks: Analysis and Solution

“…Another approach to solving the imbalanced data problem uses a recent indicator called the cross-entropy ratio (CER), 2 which was reported and evaluated in [14]. CER is defined as the ratio of the expected value of the log-likelihoods for all wrong keys to the log-likelihood of the correct key.…”

“…Another solution to the problem, cross entropy ratio (CER) was reported in [14]. CER is the ratio of the averaged negative log likelihood (NLL) of all wrong keys to the NLL of the correct one.…”

“…CER is the ratio of the averaged negative log likelihood (NLL) of all wrong keys to the NLL of the correct one. It was found in [14] that profiling attacks always succeed if the CER is less than 1 and the number of available traces is sufficient during the attack phase. In addition, [14] experimentally demonstrated that CER could solve the ineffectiveness of the performance metrics.…”

“…It was found in [14] that profiling attacks always succeed if the CER is less than 1 and the number of available traces is sufficient during the attack phase. In addition, [14] experimentally demonstrated that CER could solve the ineffectiveness of the performance metrics. However, it is still unclear why CER is effective for imbalanced data because the attack assumption used in [14], namely, independence between the intermediate values of the correct key and those of other key candidates, does not hold in general [15].…”

“…Thus, while the imbalanced data problem has been solved experimentally in [12] and [14], no analytical explanation of why the conventional solutions are effective has yet been provided, which makes the accurate assessment of DL-based SCA threats and development of effective countermeasures difficult.…”