Flow-Based Encrypted Network Traffic Classification Using Random Forest for Network Access Control

Authors

  • Wahyu Isnan Universitas Negeri Padang
  • Yulia Fatmi Universitas Negeri Padang
  • Resmi Darni Universitas Negeri Padang
  • Vikri Aulia Universitas Negeri Padang

DOI:

https://doi.org/10.24076/intechnojournal.2026v8i1.2922

Keywords:

Encrypted Network Traffic, Flow-Based Features, Random Forest, Network Traffic Classification, Network Access Control

Abstract

Encrypted network communications reduce the effectiveness of payload based traffic identification and complicate the translation of traffic analysis into network access control decisions. This study evaluates a payload independent workflow that connects flow based multiclass classification with administrator triggered, time limited firewall enforcement. The experiment used the public ISCXVPN2016 benchmark. After removing 18,719 duplicate records, 40,987 unique flows remained; all 23 available numerical flow features were retained without feature selection or normalization. A Random Forest classifier with 300 trees was selected using five-fold cross-validated grid search on a stratified 80% training partition and evaluated on an independent 8,198 sample test set covering 14 VPN and non-VPN traffic classes. The model achieved 88.01% accuracy, 88.00% weighted precision, 88.01% weighted recall, and 87.97% weighted F1-score. It exceeded the strongest reproduced baseline, K-Nearest Neighbors, by 16.09 percentage points in accuracy and 16.29 percentage points in weighted F1-score. Supplemental five-fold evaluation produced a mean accuracy of 88.05% with a 0.36 percentage point standard deviation. The trained classifier was integrated with a web application in which administrators review predicted flows and initiate temporary MikroTik RouterOS rules. All 30 temporary blocking entries observed in the evaluation database reached the Unblocked state with recorded timestamps, demonstrating rule lifecycle traceability at the database level. The findings show that Random Forest can provide competitive flow based classification while supporting an auditable, human controlled access control workflow; however, device level reliability and cross dataset generalizability require further validation.

References

[1] M. Thomson and S. Turner, "RFC 9001: Using TLS to Secure QUIC," 2021. [Online]. Available: https://www.rfc-editor.org/info/rfc9001

[2] M. Bishop, "RFC 9114: HTTP/3," 2022. [Online]. Available: https://www.rfc-editor.org/info/rfc9114

[3] A. Sharma and A. H. Lashkari, "A survey on encrypted network traffic: A comprehensive survey of identification/classification techniques, challenges, and future directions," Feb. 01, 2025, Elsevier B.V. doi: 10.1016/j.comnet.2024.110984.

[4] A. Azab, M. Khasawneh, S. Alrabaee, K. K. R. Choo, and M. Sarsour, "Network traffic classification: Techniques, datasets, and challenges," Digital Communications and Networks, vol. 10, no. 3, pp. 676–692, Jun. 2024, doi: 10.1016/j.dcan.2022.09.009.

[5] N. Wickramasinghe, A. Shaghaghi, G. Tsudik, and S. Jha, "SoK: Decoding the Enigma of Encrypted Network Traffic Classifiers," in 2025 IEEE Symposium on Security and Privacy (SP), Los Alamitos, CA, USA: IEEE Computer Society, May 2025, pp. 1825–1843. doi: 10.1109/SP61157.2025.00165.

[6] Z. Wang, Y. Yang, and Y. Wang, "A Survey of Encrypted Traffic Classification: Datasets, Representation, Approaches and Future Thinking," in 2024 IEEE/ACIS 24th International Conference on Computer and Information Science (ICIS), 2024, pp. 113–120. doi: 10.1109/ICIS61260.2024.10778376.

[7] R. Bozkir, M. Cicioglu, A. Calhan, and C. Togay, "A new platform for machine-learning-based network traffic classification," Comput. Commun., vol. 208, pp. 1–14, 2023, doi: https://doi.org/10.1016/j.comcom.2023.05.010.

[8] R. T. Elmaghraby, N. M. Abdel Aziem, M. A. Sobh, and A. M. Bahaa-Eldin, "Encrypted network traffic classification based on machine learning," Ain Shams Engineering Journal, vol. 15, no. 2, p. 102361, Feb. 2024, doi: 10.1016/J.ASEJ.2023.102361.

[9] P. Luo, J. Chu, and G. Yang, "IP packet-level encrypted traffic classification using machine learning with a light weight feature engineering method," Journal of Information Security and Applications, vol. 75, p. 103519, 2023, doi: https://doi.org/10.1016/j.jisa.2023.103519.

[10] A. O. Salau and M. M. Beyene, "Software defined networking based network traffic classification using machine learning techniques," Sci. Rep., vol. 14, no. 1, p. 20060, 2024, doi: 10.1038/s41598-024-70983-6.

[11] J.-K. Lee, T. Hong, and G. Lee, "AI-Based Approach to Firewall Rule Refinement on High-Performance Computing Service Network," Applied Sciences, vol. 14, no. 11, 2024, doi: 10.3390/app14114373.

[12] UNB Canadian Institute for Cybersecurity, "VPN-nonVPN dataset (ISCXVPN2016)," University of New Brunswick. Accessed: Apr. 12, 2026. [Online]. Available: https://www.unb.ca/cic/datasets/vpn.html

[13] Scikit-learn Developers, "RandomForestClassifier," Scikit-learn Developers. Accessed: Apr. 20, 2026. [Online]. Available: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html

[14] "GridSearchCV — scikit-learn 1.9.0 documentation." Accessed: Jul. 22, 2026. [Online]. Available: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html

[15] MikroTik, "API | RouterOS Manual." Accessed: Jul. 10, 2026. [Online]. Available: https://manual.mikrotik.com/docs/developer-guides/api/

[16] "classification_report — scikit-learn 1.9.0 documentation." Accessed: Jul. 22, 2026. [Online]. Available: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html

Downloads

Published

2026-07-31

Issue

Section

Articles