^1,2 Lecturer, Computer Science and Engineering (CSE), Bangladesh Army University of Engineering & Technology (BAUET).
³ Lecturer, Department of Computer Science and Engineering, Asian University of Bangladesh, Bangladesh.
⁴ Additional Director, ICT CELL, University of Dhaka, Bangladesh.
⁵ Software Engineer, Hajvery University, Lahore.

Full Paper PDF:

Journal: KZYJC | Volume: 40, Issue 10, 2025

This study aimed to evaluate the statistical reliability of machine learning–based intrusion detection models in real-time high-traffic network environments. Traditional offline evaluations often overestimate model performance due to random data partitioning, while temporal and real-time assessments provide more operationally relevant insights. A quantitative experimental design was employed, using three benchmark intrusion detection datasets: CICIDS2017, UNSW-NB15, and CSE-CIC-IDS2018. Models evaluated included Random Forest, Gradient Boosting (XGBoost), Support Vector Machine, Deep Neural Networks, and Autoencoder-based anomaly detectors. Data preprocessing preserved temporal order, handled missing values, removed redundant features, and scaled numerical variables. Real-time streaming simulations with sliding windows captured performance under concept drift and high-traffic conditions. Performance metrics included accuracy, precision, recall, F1-score, AUC, Brier Score, and Expected Calibration Error, with statistical reliability assessed via bootstrap confidence intervals and drift detection. Offline evaluation showed that XGBoost achieved the highest accuracy (0.989) and AUC (0.995), followed by Random Forest (accuracy 0.985, AUC 0.992). Real-time evaluation revealed lower mean F1-scores and higher variance, with XGBoost maintaining the highest mean F1-score (0.956 ±0.015) and lowest Brier Score (0.033) and ECE (2.9%), indicating superior calibration and stability. SVM and Autoencoder displayed the highest sensitivity to drift, with 11–13 ADWIN alerts and increased performance variability. Bootstrap confidence intervals further highlighted that offline superiority does not always translate to real time reliability. It is concluded that the study demonstrates that statistically robust evaluation, including temporal validation, uncertainty quantification, and real-time simulation, is essential for assessing the operational effectiveness of machine learning intrusion detection systems. XGBoost and ensemble models showed superior stability and calibration, whereas SVM and Autoencoder were less reliable under dynamic high-traffic conditions. These findings emphasize the need for adaptive learning, drift monitoring, and statistically rigorous evaluation in real-world cybersecurity applications.