Machine learning in cyber security: a structured approach

Published: Friday, 25 March 2022 09:47

Machine learning (ML) and artificial intelligence (AI) have become frequent buzzwords in the cyber security space. Security teams have an urgent need for more automated methods for detecting threats and malicious user activity and ML offers a better future. Melissa Ruzzi offers some advice for how to introduce it to your organization.

Cyber security is undergoing massive shifts in technology and operations and data science is a key element driving these future innovations. ML can play a vital role to capture insights from data within the cyber security space.

With cyber attacks becoming more widespread, sophisticated, and targeted, automation is becoming a vital tool for overwhelmed security teams. This is because most defence measures are not infallible, and many of today’s detection methods rely on manual investigation and decision making by an analyst to find advanced threats, malicious user behaviour, and other serious risks. Machine learning offers far faster results than humans can deliver in recognising and predicting certain types of patterns. 

To harness the automated innovation of ML, security teams need to understand the most suitable opportunities for implementing these technologies. Deploying ML correctly is key to being able to obtain a meaningful impact in improving an organization’s ability to detect and respond to emerging and ever-evolving cyber threats.

Machine learning and the attack surface 

The threat surface has increased exponentially due to the expansion of mobile devices, cloud storage, teleworking, distance learning, and the Internet of Things - all of which also contribute to the increase in the number of suspicious activities that do not necessarily relate to threats. The challenge is amplified by a large number of events deemed suspicious flagged by most security monitoring tools. Teams are finding it increasingly difficult to keep up with analysing suspicious activity and are finding it harder to identify emerging threats amongst a saturated threat landscape.

This is where ML comes in. From a security professional’s perspective, the need for ML and AI is strong. They’re looking for ways to automate the task of detecting threats and flagging malicious behaviour. Moving away from manual methods will free up time and resources to enable security teams to focus on other tasks. With ML, they can make use of technologies beyond deterministic rules-based approaches that require prior knowledge of fixed patterns.

Achieving data-driven IT security

ML has the ability to deliver strong capabilities when it comes to recognising certain types of patterns. These tools can offer more advanced detection than manual investigations. ML offers a more comprehensive analysis by identifying typical patterns of activity within an IT infrastructure and detecting anomalous behaviours that could signal an attack.

The effectiveness of ML relies on having access to large sets of high-quality, rich, structured data that capture the different activities happening across numerous endpoints. One of the biggest challenges in applying ML in the cyber security domain is the data ingestion, enrichment and processing needed to feed the models. Not all endpoint vendors provide the same content and the same context mapping in their logs which adds complexity in the data processing step, requiring vast domain expertise to do it correctly.

The next big challenge in ML is to choose the most appropriate algorithm and framework that will deliver the needed added value in an optimised matter. Not all available ML algorithms are appropriate to be applied in the cyber security domain due to the uniqueness of the problem ML is trying to solve here. Domain expertise combined with ML expertise is key to creating useful ML models for cyber security.

Analysing suspicious anomalous activity through ML enhances the chances organizations have of identifying and stopping emerging threats before threat actors can achieve their intended results.

Suspicious activity detection is a critical area of security that can benefit from ML. User and entity behaviour analytics (UEBA) is a perfect application for machine learning given that one of the top initial access of attacks happens via exploitation of user’s access. ML can take many more factors into consideration than humans can when looking at potential threats, and it can do so much faster.

Automated intelligent technologies have a lot to offer to security teams looking to evolve the ways they detect and mitigate growing cyber threats. Today’s security products cannot fully automate the security operations centre (SOC) and completely eliminate the need for security analysts and incident responders. However, the deployment of ML technologies can streamline and increase efficiency in critical processes to compensate for the increasing need for human responders and ease pressures on security.

Preparing for an AI powered future

ML can enable security teamsto be better, smarter, and faster by having advanced analytics at its fingertips to solve real problems - like using ML UEBA for detecting user-based threats.

The transformation of security operations through ML is an emerging approach and data-driven capabilities will continue to evolve over the coming years. For organizations to secure their future against a growing threat surface, now is the time to understand how these technologies can be deployed to achieve greater threat detection and protection outcomes.

The author

Melissa Ruzzi, Sr. Technical Product Manager, LogRhythm