Open Access iconOpen Access

ARTICLE

crossmark

An Efficient Way to Parse Logs Automatically for Multiline Events

Mingguang Yu1,2, Xia Zhang1,2,*

1 School of Computer Science and Engineering, Northeastern University, Shenyang, 110169, China
2 Neusoft Corporation, Shenyang, 110179, China

* Corresponding Author: Xia Zhang. Email: email

Computer Systems Science and Engineering 2023, 46(3), 2975-2994. https://doi.org/10.32604/csse.2023.037505

Abstract

In order to obtain information or discover knowledge from system logs, the first step is to perform log parsing, whereby unstructured raw logs can be transformed into a sequence of structured events. Although comprehensive studies on log parsing have been conducted in recent years, most assume that one event object corresponds to a single-line message. However, in a growing number of scenarios, one event object spans multiple lines in the log, for which parsing methods toward single-line events are not applicable. In order to address this problem, this paper proposes an automated log parsing method for multiline events (LPME). LPME finds multiline event objects via iterative scanning, driven by a set of heuristic rules derived from practice. The advantage of LPME is that it proposes a cohesion-based evaluation method for multiline events and a bottom-up search approach that eliminates the process of enumerating all combinations. We analyze the algorithmic complexity of LPME and validate it on four datasets from different backgrounds. Evaluations show that the actual time complexity of LPME parsing for multiline events is close to the constant time, which enables it to handle large-scale sample inputs. On the experimental datasets, the performance of LPME achieves 1.0 for recall, and the precision is generally higher than 0.9, which demonstrates the effectiveness of the proposed LPME.


Keywords


Cite This Article

APA Style
Yu, M., Zhang, X. (2023). An efficient way to parse logs automatically for multiline events. Computer Systems Science and Engineering, 46(3), 2975-2994. https://doi.org/10.32604/csse.2023.037505
Vancouver Style
Yu M, Zhang X. An efficient way to parse logs automatically for multiline events. Comput Syst Sci Eng. 2023;46(3):2975-2994 https://doi.org/10.32604/csse.2023.037505
IEEE Style
M. Yu and X. Zhang, "An Efficient Way to Parse Logs Automatically for Multiline Events," Comput. Syst. Sci. Eng., vol. 46, no. 3, pp. 2975-2994. 2023. https://doi.org/10.32604/csse.2023.037505



cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 799

    View

  • 420

    Download

  • 1

    Like

Share Link