Machine Learning-Based Malware Detection: A Comparative Study of Random Forest, Decision Tree, KNN, and Linear SVM
DOI:
https://doi.org/10.54536/ajaset.v10i2.8028Keywords:
Cybersecurity, Decision Tree, Malware Detection, Machine Learning, Random Forest, PE Files Static Analysis, Support Vector MachineAbstract
The growing prevalence of malware presents a critical threat to cybersecurity, causing substantial financial and operational damage to organizations worldwide. Traditional signature-based detection approaches are increasingly insufficient against polymorphic and zero-day threats. This paper presents a comprehensive comparative study of four machine learning (ML) algorithms — Random Forest (RF), Decision Tree (DT), K-Nearest Neighbor (KNN), and Linear Support Vector Machine (SVM) — for malware detection using static feature analysis on Portable Executable (PE) files. Experiments were conducted on a combined dataset derived from Drebin-215 and Malgenome-215 containing 18,830 instances with 208 features. A stratified 10-fold cross-validation with GridSearch CV hyperparameter tuning was employed. Evaluation metrics include accuracy, precision, recall, F1-score, and Area Under the ROC Curve (AUC). Results demonstrate that Random Forest achieves the highest performance with a test accuracy of 96.3%, F1-score of 0.947, and AUC of 0.993, outperforming all other classifiers and establishing it as the optimal algorithm for static malware detection tasks.
Downloads
References
Aslan, O. A., & Samet, R. (2020). A comprehensive review on malware detection approaches. IEEE Access, 8, 6249–6271. https://doi.org/10.1109/ACCESS.2019.2963724
Belaoued, M., & Mazouzi, S. (2015). A real-time PE-malware detection system based on chi-square test and PE-file features. In Proceedings of the International Conference on Cloud Computing and Intelligence Systems (CIIA).
Botacin, M., de Geus, P. L., Grégio, A., & Ceschin, F. (2022). HEAVEN: A hardware-enhanced antivirus engine to accelerate real-time, signature-based malware detection Expert Systems with Applications, 201, Article 117083.
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
Choi, S., Jang, S., Li, Y., & Kim, J. (2017). Malware detection using malware images and deep learning techniques. In Proceedings of the IEEE International Conference on Advanced Communications Technology (ICOIN).
Gandotra, E., Bansal, D., & Sofat, S. (2020). Malware detection using machine intelligence. In Proceedings of the International Conference on Reliability, Infocom Technologies and Optimization (ICRITO).
Gibert, D., Mateu, C., & Planes, J. (2020). The rise of machine learning for detecting and classifying malware: Research developments, trends, and challenges. Journal of Network and Computer Applications, 153, Article 102526.
Islam, F., Jamil, A., & Momen, S. (2017). Evaluation of machine learning method for Android malware detection using static features. In Proceedings of the IEEE International Conference on Computer and Information Technology.
Narudin, F. A., Feizollah, A., Anuar, N. B., & Awang, R. (2016). Evaluation of machine learning classifiers for mobile malware detection. Soft Computing, 20(1), 343–357.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
Prusty, S., Patnaik, S., & Dash, S. K. (2022). SKCV: Stratified Kfold cross-validation on ML classifiers for predicting cervical cancer. Frontiers in Nanotechnology, 4, Article 914041.
Rana, M. S., Rahman, M. M., & Rahman, M. A. (2018).Evaluation of tree-based machine learning classifiers for Android malware detection. In Proceedings of the IEEE International Conference on Computer and Information Technology (ICCIT).
Ravi, V., Alazab, M., Srinivasan, S., & Venkatraman, S. (2022). A multi-view attention-based deep learning framework for malware detection in smart healthcare systems. Computer Communications, 195, 73–81.
Shabtai, A., Moskovitch, R., Elovici, Y., & Glezer, C. (2009). Detection of malicious code by applying machine learning classifiers on static features: A state-of-the-art survey. Information Security Technical Report, 14(1), 16–29.
Shukla, M., Sharma, A., & Singh, K. (2019). A reliable binary classifier for malware detection. In Proceedings of the IEEE International Conference on Computer and Information Technology (ICCIT).
Usman, N., Usmani, S., Khan, F. R., & Shafi, I. (2021). A dynamic features-based forensic model for threat action prediction. IEEE Access, 9, 12345–12358.
Zhou, Y., & Jiang, X. (2012). Dissecting Aandroid malware: characterization and evolution. In Proceedings of the IEEE Symposium on Security and Privacy (S&P).
