A Distributed System Fault Diagnosis System based on Machine Learning

Abstract

Now days distributed system becomes the mainstream system of information storage and processing. Compared with traditional systems, distributed systems are larger and more complex.However, the average probability of failure is higher and the difficulty, complexity of operation and maintenance are greatly increased. Therefore, it is necessary to use efficient methods to diagnose the system. Our aim is to use the trained model to diagnose the fault data of the distributed system, so we can obtain as high diagnostic accuracy as possible, and create a web side for users to use. The technique we proposed uses the integrated learning approach of Stacking to model the superposition of the raw data. To realize this, we trained with a dataset of 10,000 pieces of data and assessed accuracy every once in a while. Our best training results are about 80.69% accurate and can be used on the web side. By training data sets and analyzing distributed system faults with Stacking technology, a model with a test accuracy of 80.69% was obtained. Through this model and the web platform we built, the fault of distributed system can be diagnosed, and the diagnosis results are better than other models.