Anomaly detection and classification
Anomaly detection identifies patterns in a given dataset that do not conform to an established normal behavior. The detected patterns are called anomalies and often translated to critical and actionable information in several application domains since they deviate from their normal behavior. Anomalies are also referred to as outliers, deviation, peculiarity, etc.
For example, in online banking, anomaly detection is used to identify abnormal transactions in users accounts, to protect them from fraud.

Overview of anomaly detection
The anomaly detection problem, in its most general form, is not easy to solve. In fact, most of the existing anomaly detection techniques solve a specific formulation (instance) of the problem. The formulation is induced by various factors such as nature of the data, availability of labeled data, type of anomalies to be detected, etc. Often, these factors are determined by the application domain in which the anomalies have to be detected.
Usually, in addition to the challenge of detecting anomalies in a dataset, the analyzed data is also high-dimensional (meaning data that requirse more than three dimensions to be represented). High-dimensional data is difficult to analyze and interpret. Since the data in most modern systems can be described by hundreds and even thousands of parameters (features), then the dimensionality of the data is very high and its processing becomes impractical.
"Curse of dimensionality" is associated with high-dimensional data. This is due to the fact that as the dimensionality of the input data space increases, it becomes exponentially more difficult to process and analyze the data. Furthermore, adding more dimensions can increase the noise, and hence the error and in certain situations, the number of observations is insufficient to produce satisfactory dimensionality reduction.
High-dimensional data is incomprehensible to understand, to draw conclusions from or to find anomalies that deviate from their normal behavior. Although anomaly detection identifies the anomalies in the system, it lacks the characterization of these deviations. This characterization is crucial since it provides a better understanding of the anomalies and it enables a more accurate classification of them. Anomaly characterization is a subject of recent researches. It aims at understanding the statistical, temporal or spatial behavior of the anomalies in order to characterize them and, as a result, provide a more accurate and sophisticated anomaly detection.
Scientists and reseraches in Brainstorm Private Consulting have been specializing in the development of effective anomaly detection and classification techniques in high-dimensional data for over 15 years and have applied these in many application areas.

How Businesses can benefit from Anomaly Detection
Businesses in all sectors can benefit from anomaly detection and classification. Data collected and stored in databases, including data archived in data warehouses, is data representing some real world process or processes. Anomalies and outliers which exist in the real world process will be captured with the data collected. The application of the appropriate techniques to identify and detect these anomalies can lead to new knowledge about the data and hence the real world process.
Anomaly detection is applicable in a variety of domains, such as intrusion detection, fraud detection, fault detection, system health monitoring, event detection in sensor networks, and detecting eco-system disturbances.

Case studies
Please refer to the Solutions page for Case Studies describing applications of various anomaly detection and classification techniques with client data. Please refer to the Training page for our professional Anomaly Detection and Classification training.