Machine Learning-Based Intrusion Detection: Comparative Analysis of Accuracy, False Positive Rate, and Processing Time

Abstract

In this paper the authors analyze machine learning intrusion detection models on the CICIDS2017 dataset. The algorithms tested were: Random Forest, SVM, KNN, CNN. Random Forest had a 98.7% accuracy rate, 1.2% false positive rate, and used 42 ms of computing time per sample. SVM achieved the accuracy of 96.4 percent, false positive rate of 2.8 percent and the processing time of 67ms. KNN achieved 94.1 percent accuracy, 4.5 percent false positive rate and 89 ms processing time. CNN got the best accuracy of 99.2 percent, but with a 1.5 percent false positive rate, and a processing time of 115 ms, in compromise of 2.7 to 1-time cost compared to Random Forest. The feature selection method reduced dimensionality by 35% and model overall speed by 28%, without influencing the model accuracy. The ensemble methods were 22 percent better at detecting zero-day attacks than single models. Some of the limitations are dataset bias and not being able to test networks in real-time. The results show that there is a trade-off between the accuracy and the latency; in particular, Random Forest achieves the best ratio of 1.8 between the performance and its cost in an enterprise deployment.