The concept of Security Incident and Event Management (SIEM) is not new. Indeed, systems designed to collect and aggregate event logs from servers and infrastructure have become an established element of enterprise IT administration. But there are two major problems with current SIEM systems.
First, the number of alerts being generated is increasing exponentially. Depending on the size and complexity of your network, the number of logs that need to be reviewed quickly becomes unmanageable, leading to ‘alert fatigue’.
More than half of businesses (59%) running traditional on-premise SIEM systems report that alert fatigue is a contributing factor in staff attrition."
Second, the modern IT environment itself is undergoing a radical transformation. A significant proportion of mission critical applications are now hosted (and least in part) in the cloud. The security information and event logs from these hosted platforms are extremely important, but many traditional SIEM systems are unable to monitor and assess applications, a key aspect of the hybrid environment.
By adding Machine Learning (ML) and Artificial Intelligence (AI) capabilities to SIEM, you lay the foundation for a more effective and efficient security posture.
Using the cloud to change the game
Using a cloud-based SIEM platform like Azure Sentinel changes the game. It’s able to provide SIEM capabilities for on- and off-premise resources, including logging via API - an immediate advantage.
The system is also able to ingest, process, filter and prioritise from a broad range of sources (including applications), using AI to surface only those alerts that warrant additional investigation. The scalable nature of the cloud means that there is always sufficient computing resource available to process and action events in real time.
So far so good. But by allowing ML to do the heavy lifting, you can increase efficiency even further.
ML increases anomalous event detection potential
ML is best applied to solving complex problems by observing incoming data and identifying patterns and trends that are not immediately obvious. The enormous influx of event logs makes SIEM an ideal candidate for ML.
Connected to event logs from local and cloud-based systems, the ML engine can establish a baseline reading of your normal operations. Every new incoming event is then processed and compared to the baseline to ensure they are within the established parameters of normality. By defining “normal” the ML algorithm can immediately identify an anomalous event the moment it occurs, which triggers a pre-defined, automated “playbook” that defines the next step of investigation.
A SIEM platform with built-in ML, such as Azure Sentinel, can establish a baseline of normality, to enable the accurate detection of genuine threats amongst the millions of general event logs generated, and only surfacing those that require attention. Cutting through the “white noise” like this is an essential step towards reducing overload and alert fatigue.
Knowing that they are prioritising higher risk alerts will help to keep the security team motivated and focused and help to improve morale and employee retention.
Being overloaded with incoming events inevitably leads to important issues being missed on the first pass. And the second. And the third.
The baseline of normality is used to assess incoming event logs automatically, alerting when an anomalous event is detected. This widens visibility into your IT estate, using additional tools such as User and Entity Behaviour Analytics (UEBA), to ensure that unusual events are captured and escalated the first time around.
Low-fidelity issues are particularly hard to identify, particularly when they only manifest intermittently. The ML engine, combined with AI, not only analyses new events as they are raised, but also historical logs, looking for unusual trends or opportunities for improvement.
This long-term view can help to highlight intermittent events, alerting the security team when another incident is triggered. The system can also surface all similar/related historical events so that analysts have more relevant information available as they work to resolve the issue.
82% of security professionals said that understanding the root cause of an incident is crucial in order to make improvements to their security posture."
Extended logging support
A next generation platform like Azure Sentinel, with built-in ML, will extend your SIEM capabilities beyond servers and infrastructure, applying the same powerful ML analysis to application and service monitoring too. This ensures that your entire IT estate is being monitored and analysed for incidents – automatically.
As the name suggests, UEBA is focused on user behaviour (or that of other entities) and works in a similar way to SIEM – collating data and event information, and then using ML algorithms to surface trends and alert the team when anomalous activity occurs. The difference is that UEBA tracks behaviour data and events, focusing on threats inside your environment.
The principle is quite simple – even though it’s relatively straight forward to steal someone’s username and password, being able to impersonate them by imitating their behaviour is a lot harder. So, when that person logs into the system and their behaviour varies from what’s considered ‘normal’, the UEBA ‘alarms’ start ringing.
A platform like Azure Sentinel will analyse all the relevant data sources and build baseline behavioural profiles of all the entities across your organisation, from users to IP addresses and applications. It can then identify anomalous activity compared to that baseline and help you assess whether a compromise has occurred. According to Microsoft, Azure Sentinel can also distinguish the relative sensitivity of particular assets and evaluate the ‘blast radius’ (potential impact) of any given compromised asset, helping you further prioritise your incident handling and response.
With organisations having increasingly large perimeters and complex infrastructure, including extended remote working and ‘bring your own device’ (BYOD) policies, no security strategy will be 100% effective. This means a level of proactivity is required to identify and minimise threats and breaches as soon as possible. UEBA enables you to detect a multitude of different threats and should complement your preventative measures to enhance your overall security posture.
AI turns insight into action
In terms of IT security, insights are helpful – but action is even more important. By introducing AI into your SIEM operations you can shorten mean time to detection (MTTD).
For example, Azure Sentinel adds AI to its ML-powered SIEM platform through the Fusion technology. Fusion monitors events from the SIEM log processor, performing additional analysis and using its SOAR capabilities to provide smart automation.
Finely tuned threat detection
Many SIEM platforms rely on what is known, either the experience of security employees or a pre-loaded list of signatures that indicate a security issue. Azure Sentinel, on the other hand, utilises ML to establish a baseline of normal activity, bespoke to each estate, which helps detect the “unknown” threats traversing your estate.
Azure Sentinel then takes this a step further by incorporating AI into threat detection in the form of its Fusion technology. Fusion is used to piece together anomalous activity and alerts from disparate systems and data sources that show signs of a multistage attack, which may indicate a breach. They are then prioritised as high “red” incidents to bring them to the attention of the security team.
According to Microsoft, Fusion processed 50 billion alerts from across the Azure Sentinel platform in December 2019[i]. Of these, 111 were identified as incident candidates (yellow), before further AI-powered analysis narrowed the list to just 25 high fidelity (red) incidents.
Microsoft Fusion AI processed 50 billion alerts in one month, to identify just 25 high-risk security events. That’s 2,000,000,000 events per incident."
Automating responses to reduce mean time to resolution
As well as being platform-wide, ML and AI are present at the individual instance level. This allows for even more granular detection of potential issues coming from your infrastructure and applications.
ML and AI threat detection can be enhanced and complimented in Azure Sentinel by SOAR capabilities that undertake smart, automated actions based on pre-defined “playbooks”. For instance, the system notes a user accessing the system outside their regular logon windows and interacting with systems in an unusual / suspicious way. This event may trigger a playbook that limits account access – or disables the account entirely – to prevent further activity. The playbook also triggers a new case in the IT Service Management (ITSM) platform, forwarding all logs and details of the incident for further manual investigation.
The AI in Azure Sentinel can detect suspicious activities and, when combined with its SOAR capabilities, can begin mitigation far quicker than your security team. By acting quickly, the system can contain and limit damage during a security breach and help to accelerate mean time to remediation by providing the security team with the information they need to finalise protective measures.
Reducing time to identify and remediate issues typically reduces the severity of the breach and helps to contain costs. Containing a breach in less than 200 days saves an average of $1 million over an incident that lasts any longer.
Manual event log processing has not been a viable proposition for years. The security team is already overloaded without having to sort and filter tens of thousands of records to identify other potential issues.
Integrating ML and AI into SIEM dramatically improves incident detection. ML never gets tired, bored or overwhelmed, instantly detecting threats that may have been missed as the result of alert fatigue.
AI adds value to SIEM, by cutting through the white noise, surfacing only those alerts which have a high probability of being a genuine threat. The addition of intelligent playbooks and autonomous action further accelerates incident response, helping to reduce potential impact and the security team’s workload.
ML and AI are not only the future of SIEM, they’re a key toolset in helping your company to meet the security challenges of the modern hybrid operating environment.