“There are high impact security breaches in spite of using state of the art security technologies and services. The number of technologies implemented by organisations to address new and emerging threats has increased over the last three to four years. However, each technology generates its own set of events, alerts and data. Hence the number of events to act on has increased exponentially,” warned Ummer.
Most organisations with breaches have received alerts related to the breach. Ummer believes these breaches were overlooked, as analysts were occupied with analyzing and responding to other events. A fact compounded by each of these security technologies having significant number of false positives, making it complex to detect, prioritize and act on the right event. “The result is missed attacks and security breaches despite high CAPEX and OPEX,” Ummer stated. “It is obvious that real attack alerts get lost in the sea of other alerts.”
The use of statistical inference, machine learning and visualization techniques on security data has become a key component of information security strategy. Security intelligence is the fusion of statistical models, machine learning, visualization and big data, and provides better analysis.
The current approach to threat detection is to analyse security event data by applying rules. Dashboards are then viewed in parallel with a manual analysis. Ummer believes intelligence driven analysis is the way forward, which applies a contextual and historical element to the data through statistics and predictive algorithms (machine learning).
The use of a K-means clustering algorithm to look at distinct events, volume of suspicious and compromised events and the weight for categories results in alerts from SIEM on an hourly basis. This technique succeeded in highlighting specific details related to the breach e.g. attacker IP.
“Machine learning comes through intelligent and targeted analysis which means advanced attacks can be explored and patterns revealed. This makes breaches less likely and hackers less successful,” concludes Ummer.
