How ML and AI have enhanced incident detection and response management

The concept of Security Incident and Event Management (SIEM) is not new. Indeed, systems designed to collect and aggregate event logs from servers and infrastructure have become an established element of enterprise IT administration. But there are two major problems with current SIEM systems.

First, the number of alerts being generated is increasing exponentially. Depending on the size and complexity of your network, the number of logs that need to be reviewed quickly becomes unmanageable, leading to ‘alert fatigue’.

Second, the modern IT environment itself is undergoing a radical transformation. A significant proportion of mission critical applications are now hosted (and least in part) in the cloud. The security information and event logs from these hosted platforms are extremely important, but many traditional SIEM systems are unable to monitor and assess applications, a key aspect of the hybrid environment.

By adding Machine Learning (ML) and Artificial Intelligence (AI) capabilities to SIEM, you lay the foundation for a more effective and efficient security posture.

Using a cloud-based SIEM platform like Microsoft Sentinel changes the game. It’s able to provide SIEM capabilities for on- and off-premise resources, including logging via API - an immediate advantage.

The system is also able to ingest, process, filter and prioritise from a broad range of sources (including applications), using AI to surface only those alerts that warrant additional investigation. The scalable nature of the cloud means that there is always sufficient computing resource available to process and action events in real time.

So far so good. But by allowing ML to do the heavy lifting, you can increase efficiency even further.

ML increases anomalous event detection potential

ML is best applied to solving complex problems by observing incoming data and identifying patterns and trends that are not immediately obvious. The enormous influx of event logs makes SIEM an ideal candidate for ML.

Connected to event logs from local and cloud-based systems, the ML engine can establish a baseline reading of your normal operations. Every new incoming event is then processed and compared to the baseline to ensure they are within the established parameters of normality. By defining “normal” the ML algorithm can immediately identify an anomalous event the moment it occurs, which triggers a pre-defined, automated “playbook” that defines the next step of investigation.

Alert fatigue

A SIEM platform with built-in ML, such as Microsoft Sentinel, can establish a baseline of normality, to enable the accurate detection of genuine threats amongst the millions of general event logs generated, and only surfacing those that require attention. Cutting through the “white noise” like this is an essential step towards reducing overload and alert fatigue.

Knowing that they are prioritising higher risk alerts will help to keep the security team motivated and focused and help to improve morale and employee retention.

Low-level issues

Low-fidelity issues are particularly hard to identify, particularly when they only manifest intermittently. The ML engine, combined with AI, not only analyses new events as they are raised, but also historical logs, looking for unusual trends or opportunities for improvement.

This long-term view can help to highlight intermittent events, alerting the security team when another incident is triggered. The system can also surface all similar/related historical events so that analysts have more relevant information available as they work to resolve the issue.

As the name suggests, UEBA is focused on user behaviour (or that of other entities) and works in a similar way to SIEM – collating data and event information, and then using ML algorithms to surface trends and alert the team when anomalous activity occurs. The difference is that UEBA tracks behaviour data and events, focusing on threats inside your environment.

The principle is quite simple – even though it’s relatively straight forward to steal someone’s username and password, being able to impersonate them by imitating their behaviour is a lot harder. So, when that person logs into the system and their behaviour varies from what’s considered ‘normal’, the UEBA ‘alarms’ start ringing.

A platform like Microsoft Sentinel will analyse all the relevant data sources and build baseline behavioural profiles of all the entities across your organisation, from users to IP addresses and applications. It can then identify anomalous activity compared to that baseline and help you assess whether a compromise has occurred. According to Microsoft, Sentinel can also distinguish the relative sensitivity of particular assets and evaluate the ‘blast radius’ (potential impact) of any given compromised asset, helping you further prioritise your incident handling and response.

With organisations having increasingly large perimeters and complex infrastructure, including extended remote working and ‘bring your own device’ (BYOD) policies, no security strategy will be 100% effective. This means a level of proactivity is required to identify and minimise threats and breaches as soon as possible. UEBA enables you to detect a multitude of different threats and should complement your preventative measures to enhance your overall security posture.

Finely tuned threat detection

Many SIEM platforms rely on what is known, either the experience of security employees or a pre-loaded list of signatures that indicate a security issue. Microsoft Sentinel, on the other hand, utilises ML to establish a baseline of normal activity, bespoke to each estate, which helps detect the “unknown” threats traversing your estate.

Microsoft Sentinel then takes this a step further by incorporating AI into threat detection in the form of its Fusion technology. Fusion is used to piece together anomalous activity and alerts from disparate systems and data sources that show signs of a multistage attack, which may indicate a breach. They are then prioritised as high “red” incidents to bring them to the attention of the security team.

According to Microsoft, Fusion processed 50 billion alerts from across the Microsoft Sentinel platform in December 2019[i]. Of these, 111 were identified as incident candidates (yellow), before further AI-powered analysis narrowed the list to just 25 high fidelity (red) incidents.

Automating responses to reduce mean time to resolution

As well as being platform-wide, ML and AI are present at the individual instance level. This allows for even more granular detection of potential issues coming from your infrastructure and applications.

ML and AI threat detection can be enhanced and complimented in Microsoft Sentinel by SOAR capabilities that undertake smart, automated actions based on pre-defined “playbooks”. For instance, the system notes a user accessing the system outside their regular logon windows and interacting with systems in an unusual / suspicious way. This event may trigger a playbook that limits account access – or disables the account entirely – to prevent further activity. The playbook also triggers a new case in the IT Service Management (ITSM) platform, forwarding all logs and details of the incident for further manual investigation.

The AI in Microsoft Sentinel can detect suspicious activities and, when combined with its SOAR capabilities, can begin mitigation far quicker than your security team. By acting quickly, the system can contain and limit damage during a security breach and help to accelerate mean time to remediation by providing the security team with the information they need to finalise protective measures.

Reducing time to identify and remediate issues typically reduces the severity of the breach and helps to contain costs. Containing a breach in less than 200 days saves an average of $1 million over an incident that lasts any longer.

Conclusion

Manual event log processing has not been a viable proposition for years. The security team is already overloaded without having to sort and filter tens of thousands of records to identify other potential issues.

Integrating ML and AI into SIEM dramatically improves incident detection. ML never gets tired, bored or overwhelmed, instantly detecting threats that may have been missed as the result of alert fatigue.

AI adds value to SIEM, by cutting through the white noise, surfacing only those alerts which have a high probability of being a genuine threat. The addition of intelligent playbooks and autonomous action further accelerates incident response, helping to reduce potential impact and the security team’s workload.

ML and AI are not only the future of SIEM, they’re a key toolset in helping your company to meet the security challenges of the modern hybrid operating environment.

Introducing ML into your SIEM operations solves several problems

UEBA | Next Level Protection