TY - GEN

T1 - Unsupervised Anomaly Detection on Distributed Log Tracing through Deep Learning

AU - Khudyakov, Daniil A.

AU - Yakhyaeva, Gulnara E.

N1 - Conference code: 25

PY - 2024

Y1 - 2024

N2 - Large modern software systems have a distributed microservice architecture. Such systems are built from loosely connected and independently deployable components - microservices. Microservices interact with each other with the help of various protocols, forming a sequence of internal calls in order to process a request that comes to the system. This interaction is recorded in the form of logs, which are produced by services at various points of execution and contain useful debugging information. Distributed tracing links individual logs into traces. Thanks to this, it is possible to analyze all logs related to one specific request. Traces are extremely convenient for analyzing incidents that occur during system operation. This work is dedicated to the automatic analysis of log traces to identify abnormal behavior in the operation of distributed systems. The solution is based on the application of deep machine learning methods for analyzing log sequences. The logs are cleaned and vectorized, and then used to train a model based on the long short-term memory autoencoder. The solution is tested on the TraceBench open dataset. As a result, high values of precision and recall metrics were obtained.

AB - Large modern software systems have a distributed microservice architecture. Such systems are built from loosely connected and independently deployable components - microservices. Microservices interact with each other with the help of various protocols, forming a sequence of internal calls in order to process a request that comes to the system. This interaction is recorded in the form of logs, which are produced by services at various points of execution and contain useful debugging information. Distributed tracing links individual logs into traces. Thanks to this, it is possible to analyze all logs related to one specific request. Traces are extremely convenient for analyzing incidents that occur during system operation. This work is dedicated to the automatic analysis of log traces to identify abnormal behavior in the operation of distributed systems. The solution is based on the application of deep machine learning methods for analyzing log sequences. The logs are cleaned and vectorized, and then used to train a model based on the long short-term memory autoencoder. The solution is tested on the TraceBench open dataset. As a result, high values of precision and recall metrics were obtained.

KW - LSTM

KW - anomaly detection

KW - autoencoder

KW - distributed tracing

KW - log analysis

KW - microservices

KW - unsupervised deep learning

UR - https://www.scopus.com/record/display.uri?eid=2-s2.0-85201952476&origin=inward&txGid=c6d6ed79c51834788ea6813784109575

UR - https://www.mendeley.com/catalogue/740dc5f6-6174-307e-9a37-b00caefb7014/

U2 - 10.1109/EDM61683.2024.10615125

DO - 10.1109/EDM61683.2024.10615125

M3 - Conference contribution

SN - 9798350389234

T3 - International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices, EDM

SP - 1830

EP - 1833

BT - International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices, EDM

PB - IEEE Computer Society

T2 - 25th IEEE International Conference of Young Professionals in Electron Devices and Materials

Y2 - 28 June 2024 through 2 July 2024

ER -