Adversarial robustness evaluation of hybrid CNN-LSTM-transformer NIDS on evolving threats

Abstract

[EN]Current Network Intrusion Detection Systems (NIDS) often fail to detect adversarial evasion attacks, creating critical security blind spots. To address this, we propose a standardized adversarial evaluation protocol that quantifies performance degradation against Fast Gradient Sign Method (FGSM), Projected Gradient Descent (PGD), and AutoAttack ensemble attacks, establishing empirically observed performance bounds. We implemented a high-throughput hybrid architecture combining 1D-CNN, Bidirectional LSTM, and Transformer mechanisms, designed specifically to balance varying traffic dynamics and robustness. Unlike prior studies that report only clean-data accuracy, our evaluation of UNSW-NB15, CICIDS2017, and CICIoT2023 demonstrates competitive performance (e.g., strong multi-class F1 scores) while revealing robustness profiles up to an operational limit of e=0.05. Crucially, we validated our results under a temporal split using the official UNSW-NB15 train/test partition, confirming that binary detection (94.20% accuracy, 95.69% F1) generalizes under distribution shift. We further compared the proposed method with PGD-based adversarial training (PGD-AT) to quantify the robustness–accuracy trade-off. Our results advocate the use of security curves as a standard metric for NIDS validation in hostile environments.