Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Research › peer-review
Unsupervised Anomaly Detection on Distributed Log Tracing through Deep Learning. / Khudyakov, Daniil A.; Yakhyaeva, Gulnara E.
International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices, EDM. IEEE Computer Society, 2024. p. 1830-1833 (International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices, EDM).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Research › peer-review
}
TY - GEN
T1 - Unsupervised Anomaly Detection on Distributed Log Tracing through Deep Learning
AU - Khudyakov, Daniil A.
AU - Yakhyaeva, Gulnara E.
N1 - Conference code: 25
PY - 2024
Y1 - 2024
N2 - Large modern software systems have a distributed microservice architecture. Such systems are built from loosely connected and independently deployable components - microservices. Microservices interact with each other with the help of various protocols, forming a sequence of internal calls in order to process a request that comes to the system. This interaction is recorded in the form of logs, which are produced by services at various points of execution and contain useful debugging information. Distributed tracing links individual logs into traces. Thanks to this, it is possible to analyze all logs related to one specific request. Traces are extremely convenient for analyzing incidents that occur during system operation. This work is dedicated to the automatic analysis of log traces to identify abnormal behavior in the operation of distributed systems. The solution is based on the application of deep machine learning methods for analyzing log sequences. The logs are cleaned and vectorized, and then used to train a model based on the long short-term memory autoencoder. The solution is tested on the TraceBench open dataset. As a result, high values of precision and recall metrics were obtained.
AB - Large modern software systems have a distributed microservice architecture. Such systems are built from loosely connected and independently deployable components - microservices. Microservices interact with each other with the help of various protocols, forming a sequence of internal calls in order to process a request that comes to the system. This interaction is recorded in the form of logs, which are produced by services at various points of execution and contain useful debugging information. Distributed tracing links individual logs into traces. Thanks to this, it is possible to analyze all logs related to one specific request. Traces are extremely convenient for analyzing incidents that occur during system operation. This work is dedicated to the automatic analysis of log traces to identify abnormal behavior in the operation of distributed systems. The solution is based on the application of deep machine learning methods for analyzing log sequences. The logs are cleaned and vectorized, and then used to train a model based on the long short-term memory autoencoder. The solution is tested on the TraceBench open dataset. As a result, high values of precision and recall metrics were obtained.
KW - LSTM
KW - anomaly detection
KW - autoencoder
KW - distributed tracing
KW - log analysis
KW - microservices
KW - unsupervised deep learning
UR - https://www.scopus.com/record/display.uri?eid=2-s2.0-85201952476&origin=inward&txGid=c6d6ed79c51834788ea6813784109575
UR - https://www.mendeley.com/catalogue/740dc5f6-6174-307e-9a37-b00caefb7014/
U2 - 10.1109/EDM61683.2024.10615125
DO - 10.1109/EDM61683.2024.10615125
M3 - Conference contribution
SN - 9798350389234
T3 - International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices, EDM
SP - 1830
EP - 1833
BT - International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices, EDM
PB - IEEE Computer Society
T2 - 25th IEEE International Conference of Young Professionals in Electron Devices and Materials
Y2 - 28 June 2024 through 2 July 2024
ER -
ID: 60549252