Recent years have witnessed a technological renaissance driven by the rapid advancements in artificial intelligence (AI). This burgeoning field has woven itself into the very fabric of modern life, revolutionizing various sectors with its transformative capabilities. From self-driving cars navigating city streets autonomously to AI-generated art and music that rival human creativity, the impact of AI is undeniably profound.
Yet, as AI’s influence extends its reach, it’s not just redefining transportation and artistic expression; it’s also making its mark on the world of cybersecurity. While this shift promises to usher in a new era of productivity and effectiveness, it also presents a set of challenges that can keep security teams up at night. In this article, we’ll explore the flip side of the AI coin in cybersecurity, delving into the potential disadvantages and, more importantly, how to navigate them successfully.
Introduction
To level set, we’ll start with the very basics and define the terms involved:
Definition of cybersecurity
“Cybersecurity is the practice of protecting networks, devices, and data from unauthorized access or criminal use; and of ensuring confidentiality, integrity, and availability of information.1”
Definition of artificial intelligence
Artificial intelligence (AI) is the ability of machines to perform tasks that are typically associated with human intelligence, such as learning and problem-solving.2 This is a very broad definition, especially considering that humans perform a wide variety of tasks. In light of that it’s important to understand that AI is not a singular technique, rather it encompasses many techniques, sub-fields, and applications, such as machine learning, natural language processing, neural networks, and more.
A brief history of AI in cybersecurity
Over the last several years, the Cybersecurity industry has exponentially increased the use of AI technology and AI algorithms in cybersecurity solutions. Due to the broad definition of AI, it’s hard to pin-point the beginnings of this trend, but what’s clear is that product leaders have been steadily finding new and innovative ways for machines to perform security tasks previously relegated to humans, such as improving cybersecurity postures, triaging alerts, investigating cyber attacks and more.
Several key milestones for AI in Cybersecurity:
- 1980s – The introduction of rule-based systems for anomaly detection (think firewall rules).
- 2000s – The introduction of Big Data which over the next decade would make it possible to store and analyst vast volumes of data.
- Late 2000s – Supervised learning algorithms for threat detection and prevention.
- 2010s
- Unsupervised learning algorithms enabled the identification of anomalous patterns and previously unknown threats.
- Deep learning enables the processing of vast amounts of data and uncover complex patterns.
- Behavioral modeling and analytics make their way into a wide variety of tools like endpoint detection, network analytics, SIEMs, and more.
- Natural language processing (NLP) techniques gain prominence, allowing for enhanced analysis of text data and the detection of social engineering attacks.
More recently, in the 2020s the buzz generated by ChatGPT and Google Bard has prompted many security vendors to add “generative AI” in the form of ChatGPT integrations, Large Language Models (LLMs) and other advanced natural language processing capabilities to their products.
Potential Disadvantages of AI in Cybersecurity
With the influx of LLMs in today’s cybersecurity products, here are some potential disadvantages to watch out for.
Expertise is often required
For solutions that leverage LLMs, often they aim to provide insights about an organization, its environment, threats, indicators of compromise (IoC), etc. by allowing users to ask questions of a data set, typically stored on a graph database, using natural language. For example, instead of creating a query, an analyst might ask the AI to “tell me everything you know about this IP address” or “is this IoC present in my environment?”. This approach basically translates the natural language question of the analysts into the specific query capable of answering their question.
While, LLMs seem like they can simplify some SecOps processes required to perform triage, investigation, and threat hunting; analysts still must understand what questions to ask, in what order, and have enough experience to interpret the results and finally convert the results into further action steps like containment and remediation. Essentially, the gains afforded by LLMs are incrementally better than previous approaches, but it does little to lower the expertise bar for would-be analysts.
Privacy Concerns
Several security vendors have opted to provide ChatGPT integration within their products instead of creating their own LLMs or AI systems. Security leaders should be wary of products with these types of integrations because ChatGPT (and similar solutions) never forgets. It indefinitely retains and trains its models based on this information and in the case of security tools it could include potentially sensitive information like proprietary data, threat analysis, IoCs, etc. This could result in sensitive corporate information being access by future ChatGPT users.
Figure 1 – an illustration of a ChatGPT user gaining access to sensitive information.
Inaccurate Results
Each AI is different but there are several concerns to watch out for in LLM based solutions, chiefly LLM bias and LLM hallucinations, which can greatly impact their usefulness in cybersecurity applications.
LLM Bias
A critical issue within LLMs, revolves around the concept of bias, which is basically an unintentional form of data poisoning. LLM bias encompasses situations where AI systems display a form of preference or discrimination, typically mirroring the biases ingrained in their training data. It’s crucial to grasp that these biases aren’t deliberate beliefs held by the AI system; instead, they are unintentional reflections of the data utilized during the training process. In some instances, LLM hallucinations can exacerbate these biases, as the AI, in its endeavor to generate contextually relevant responses, may inadvertently draw upon biased patterns or stereotypes present in its training data.
LLM Hallucinations
LLM hallucinations are another potential concern with regard to accuracy. Within the framework of LLMs, the term “hallucination” denotes a situation in which the model produces text that is inaccurate, illogical, or fictitious. There are various causes for this including incomplete or contradictory data sets and guesswork from vague or insufficiently detailed prompts. Regardless of the cause, it’s easy to see how illogical, fictional, or nonsensical responses within the context of cybersecurity could be a problem.
Figure 2 – A diagram showing several causes and types of LLM hallucinations. Image source: master of code.
Lack of Transparency
Many modern AI solutions are total “black boxes”, meaning it’s unclear to users how they work under the hood, or how their conclusions were reached. Each vendor and solution differs with regard to the detail they provide to end users, but a lack of transparency breeds questions about accuracy and trustworthiness. In cybersecurity this concern is especially pertinent because decisions that can impact the security of an organization rest upon the output of security tools.
Steps to Reduce AI-Related Risks
This article is not intended to scare security professionals away from the use of AI in cybersecurity products, but rather to highlight potential risks and to offer guidance on how teams can overcome them. This section will look at some ways security products can avoid the pitfalls listed above and provide tips for security teams looking to pick effective cybersecurity AI solutions.
Increasing transparency with detailed activity records
Security leaders should look for AI solutions that expose how conclusions were reached, what activity has been performed, what source data was used, and where it used external data sources like threat intelligence. This type of open transparency improves trust and understanding of the results generated by a particular system.
For example, if an AI’s task was to perform alert triage on security alerts coming from various security tools, it would be important to know what checks were performed, in what order, what the results of each check were, and which pieces of information led the system to conclude that something is malicious. Conversely, if something was deemed to be benign, an audit trail showing all the inspections performed can be very useful backing up that conclusion and even for providing an auditor with proof that an alert was in fact a false positive.
Figure 3 – A track record of all tests and inspections performed by an AI, along with detailed results.
Select a proprietary AI instead of ChatGPT
It’s important to select a solution that is based on a proprietary AI system developed by a security conscious vendor. Proprietary AI products are more likely to effectively streamline and automate security workloads, because they are often purpose built to solve niche, but painful problems that security teams face. Additionally, these products are less likely to be built using widely deployed consumer products like ChatGPT, Microsoft Co-pilot, and Google Bard. This prevents your security incident data and intellectual property, IoCs and more, from being forever part of the ChatGPT training data set, and thus potentially accessible by others.
AI doesn’t need to be generative to be effective
While generative AI and LLMs are all the rage right now, other types of AI exist which can be very effective at performing SecOps tasks. Many applications of AI are less susceptible to the problems of LLMs like data bias and hallucinations. For example, a cybersecurity AI system might be built specifically to triage security alerts by replicating the question and answer processes that security analysts use to evaluate alerts and conduct incident investigations.This application focuses on understanding the contents of security alerts as opposed to the instructions of humans, and thus does not need the NLP capabilities LLMs are usually adopted for.
Even without being generative, AI can be designed to effectively scrutinize security alerts by replicating the question and answer process that a human user would utilize to determine if an alert is malicious and to perform an in depth investigation and to obtain answers from data sources like MITRE ATT&CK, CIS, threat intelligence feeds, learned environment behavior and activity benchmarks for the environment in which the alert occurred.
Improve accuracy by blending techniques
The best security operations outcomes come from a combination of techniques which may include LLMs but also effective use of other forms of AI like machine learning models, heuristics, and other techniques. The order and combination of these techniques can greatly impact accuracy. For example, a behavioral model in a UEBA tool from the late 2010s might ask the question “what abnormal credential use exists within my organization?” and while this is a truly useful piece of information, it’s bound to turn up lots of false positives because users behave abnormally. Employees travel, they use VPNs to watch Netflix, they login from new places for the first time, they get new devices, transfer departments, etc. and any of these things can be enough to impact the risk score of an individual.
However, if you asked the question “what abnormal credential use exists within my organization?” from a smaller, more targeted cross section of an organization, you would almost certainly end up with better results. If a system waited to ask this question until it had already determined that a set of users had received, opened, and clicked on a phishing email and then entered their credentials into the phishing site—it would have much higher chances of finding a true positive. In this way, AI techniques and machine learning algorithms can build on each other to greatly boost their accuracy. For the purpose of improving SecOps triage and investigation, being able to connect the dots between security data sources can further improve conclusions.
Boost analyst productivity by targeting painful workloads
Effective applications of AI are often designed to tackle very specific and routine tasks. For security operations this can translate to specific tasks within each phase of the SecOps incident lifecycle where AI can provide valuable, decision ready results to analysts. For example, the question and answer replication we discussed earlier in this post can be used to determine alert maliciousness, to perform incident investigation, and to the scope and root cause of an incident. This information can be programmatically translated into a customized response plan that will help SOC analysts address each detected security issue using best practices.
Armed with a detailed summary, root cause information, and a detailed response plan, security analysts are put into a position that by the time they receive an incident, they have all the information needed to understand the incident and take action on it. This approach completely leapfrogs the need to ask LLMs for help generating queries and greatly boosts SOC productivity.
Figure 4 – an AI generated incident summary, complete with the full incident scope and root cause analysis
Figure 5 – An AI generated set of instructions for SOC analysts to perform an incident response task as part of containment or remediation.
Conclusion
Today’s cybersecurity personnel can greatly benefit from the use of modern AI cybersecurity solutions. These tools offer tangible productivity gains that can help offset the longstanding shortage of security experts available to staff SOCs. While these tools may be useful for detecting cybersecurity threats, cross referencing security information against threat intelligence tools in real-time, and helping human teams detect, investigate, and respond to cyber threats, they are not without disadvantages. Security professionals looking to make the most of these tools should avoid the following pitfalls.
Summary of Potential Disadvantages
- High entry barriers – i.e. relying on tools which only provide productivity gains to those with a high degree of security expertise.
- Privacy issues – i.e. using tools which will potentially make your corporate data or threat information available in later versions of their engine like ChatGPT.
- Inaccurate findings – i.e. using tools, especially those built on LLMs, that may be susceptible to data bias and hallucinations which could have ramifications for your security team.
- Black boxes – i.e. selecting tools that offer little to no information about what actions are taken or how conclusions were reached.
Steps to Reduce Risks
- Select a vendor that built their own AI as opposed to one that integrates with a tool like ChatGPT.
- Generative AI and NLP isn’t the only option when it comes to using AI to streamline your SOC. Boost your SOC’s productivity by finding tools that alleviate your SOC’s biggest pain points and time sinks.
- Improve accuracy by finding solutions that blend many AI techniques to improve accuracy and results.
- Focus on tools that offer transparency around the activity they take, the results, and how they came to their conclusions.
How Radiant Can help
Learn more about how Radiant Security’s AI-powered SOC co-pilot can boost your analyst productivity, help you detect more, real attacks, and reduce your SOC’s response times at https://radiantsecurtiy.ai
