Open Access

ARTICLE

SemBERT: Semantic BERT Embeddings and HDBSCAN Clustering for Unsupervised Log Parsing and Template Mining in Large-Scale Distributed Systems

Gobinda Bhattacharjee¹, Joy Dey¹, Tanjim Mahmud^1,*, Mohammad Shahadat Hossain^2,3, Karl Andersson³

1 Department of Computer Science and Engineering, Rangamati Science and Technology University, Rangamati, Bangladesh
2 Department of Computer Science and Engineering, University of Chittagong, Chittagong, Bangladesh
3 Cybersecurity Laboratory, Luleå University of Technology, Skellefteå, Sweden

* Corresponding Authors: Tanjim Mahmud. Email: ,
# A preliminary version of this work was presented at the 9th International Conference on Mobile Internet Security (MobiSec 2025, Japan), organized by the KIISC Research Group on 6G Security at the Electronics and Telecommunications Research Institute (ETRI) and hosted by the Korea Institute of Information Security and Cryptology. This manuscript substantially extends the conference version by introducing statistical robustn

Computers, Materials & Continua 2026, 89(2), 81 https://doi.org/10.32604/cmc.2026.085032

Received 04 May 2026; Accepted 07 August 2026; Issue published 15 September 2026

Abstract

Log parsing is a fundamental prerequisite for automated system monitoring, anomaly detection, and root cause analysis in large-scale distributed environments. However, existing parsing approaches often rely on heuristic rules, manually engineered features, or fixed similarity thresholds, limiting their adaptability to heterogeneous and evolving log structures. To address these challenges, this study presents SemBERT, a fully unsupervised log parsing framework that integrates semantic BERT embeddings, Incremental Principal Component Analysis (IPCA), HDBSCAN clustering, and adaptive centroid-based cluster merging for robust template mining. Unlike conventional methods that employ fixed merging criteria, SemBERT adaptively determines semantic merging thresholds according to the distribution of cluster centroids, reducing parameter sensitivity across datasets. Experiments on the LogHub HDFS benchmark demonstrated that SemBERT achieved perfect Grouping Accuracy (GA = 100.00%), Parsing Accuracy (PA = 95.82 ± 0.44%), and Template Accuracy (TA = 76.59 ± 4.49%), outperforming several classical parsers in semantic template discovery while maintaining competitive parsing performance. Repeated reproducibility experiments using three different random seed initializations (42, 123, and 999) produced identical adaptive thresholds and final template counts across all evaluated data scales, confirming the deterministic behavior and reproducibility of the proposed framework. Additional scalability experiments on randomly sampled subsets ranging from 50K to 1M log entries demonstrated stable threshold estimation and controlled template growth, highlighting the effectiveness of the adaptive semantic merging strategy. To validate the practical utility of the extracted templates, downstream anomaly detection experiments were conducted using representative machine learning, deep learning, and transformer-based models. The semantic templates generated by SemBERT provide informative representations suitable for downstream log analytics. Furthermore, experiments on the BGL dataset indicate that the framework generalizes effectively to heterogeneous logging environments through lightweight dataset-specific preprocessing. These findings suggest that SemBERT provides a scalable and semantically informed solution for unsupervised log parsing in modern distributed systems.

---

Keywords

---