Abstract: Fuel usage and costs related to it are of enormous interest, especially in aeroespacial domain. In the past engineers had to manually go through thousands of flights in order to seek the abnormal ones. Positive anomalies are to be mimicked, for negatives it’s crucial to understand the reasons to avoid such scenarios in the future, normal flights help drawing insights about typical flight behavior. However, the process is tedious, time consuming and error prone as well as it’s hardly possible for a human to eyeball so large data volume. The flight anomaly detection project we delivered for our client made it possible to limit the manual work to a minimum while increasing effectiveness. Currently engineers only have to go through a limited subset of flights, just the suspicious ones proposed by the algorithm, so they have more time to create valuable insights and focus on other tasks.
Behind the scenes, anomalies detector leverages Machine Learning algorithms, both for data quality handling and abnormalities detection itself. Reducing the volume of problematic, not useful data, is done with the aid of classifiers’ ensemble (96% accuracy). The anomalies detection itself has been solved in two ways. One approach combines clustering and Isolation Forest and is preceded with outliers removal and flights unification. The other profits from correlation with weather and sophisticated forecasting algorithms.
Ideas from this project can be transferred easily to any other domain, where there is a need for multivariate anomalies detection, especially if objects with unequal series of events are involved.
Bio: Head of Data Science at Objectivity with background in Mathematical Statistics. For about 9 years, she has been discovering the potential of data in various business domains, from medical data, through retail, HR, finance, aviation, real estate, ... She deeply believes in the power of data in every area of life. Articles’ writer, conference speaker and privately - passionate dancer.
Head of Data Science | Objectivity