Detect anomalies 5 mins ahead with 95% accuracy

Anomaly Detection

Anomaly Detection in Web Infrastructure System

Logistics planning


Anomaly Detection refers to a wide array of techniques that primarily focus on detecting anomalies in the functioning of systems, application and

processes.

Any data analyst will be aware of the term outliers. When outliers occur in a data set, it is important to determine whether they are anomalies requiring investigation.


The most common applications are in fraud detection in Credit Card / Telecom Industries.

However, with massive architecture of systems anomaly detection algorithms can be used in many applications where automated processes are involved.Imagine the sheer set of automated processes in the world. Cron jobs, daily scripts perform a wide range of activities from operational updates to revenue critical processes.


A malfunction or deviation of expected functionality can go unnoticed for a long time, causing operational issues or loss of revenue before anyone realises it.


ORMAE's anomaly detection solutions can make sure your automated processes are not failing you or your business.


Anomaly Detection in Web Infrastructure System of Retail Client

 detection

Many e-commerce portals face unexpected response time by their portals which may depend on host of reasons like excessive traffic, server downtime, … Identifying such reasons can lead to improved customer service. Identifying these anomalies is a challenging task as they occur very infrequently.


Challenges, Solution and Business Impact

ORMAE  have worked on developing an Anomaly detection algorithm of web-infrastructure system for one of the major Retail company in USA which had hundred of servers, multiple applications and data at one minute granularity.


Expertise Solutions


Image title  Identify and isolate the reasons for detecting violations in response times 


Image title  Predict violations and anomalies 5 minutes ahead of time


Image title No set definitions of anomalies . Based on the data the response time was designed of the server.


Image title Mid/Long term capacity planning to achieve revenue targets

Optimizing Operations


Huge amount of unclean and unstructured data and no easy observable pattern in data. Method is unsupervised where validation can be quite challenging.


Response time violations were not consistent in pattern, hence running pattern recognition was not feasible.


Random forest had produced the best possible result, in being able to predict the response time violation 5 minutes ahead of time with 93% accuracy