Adversarial attacks on AI models are rising: what should you do now?

Adversarial attacks on machine learning (ML) models are growing in intensity, frequency and sophistication with more enterprises admitting they have experienced an AI-related security incident.

AI’s pervasive adoption is leading to a rapidly expanding threat surface that all enterprises struggle to keep up with. A recent Gartner survey on AI adoption shows that 73% of enterprises have hundreds or thousands of AI models deployed.

HiddenLayer’s earlier study found that 77% of the companies identified AI-related breaches, and the remaining companies were uncertain whether their AI models had been attacked. Two in five organizations had an AI privacy breach or security incident of which 1 in 4 were malicious attacks.

A growing threat of adversarial attacks

With AI’s growing influence across industries, malicious attackers continue to sharpen their tradecraft to exploit ML models’ growing base of vulnerabilities as the variety and volume of threat surfaces expand.

Adversarial attacks on ML models look to exploit gaps by intentionally attempting to redirect the model with inputs, corrupted data, jailbreak prompts and by hiding malicious commands in images loaded back into a model for analysis. Attackers fine-tune adversarial attacks to make models deliver false predictions and classifications, producing the wrong output.

VentureBeat contributor Ben Dickson explains how adversarial attacks work, the many forms they take and the history of research in this area.

Gartner also found that 41% of organizations reported experiencing some form of AI security incident, including adversarial attacks targeting ML models. Of those reported incidents, 60% were data compromises by an internal party, while 27% were malicious attacks on the organization’s AI infrastructure. Thirty percent of all AI cyberattacks will leverage training-data poisoning, AI model theft or adversarial samples to attack AI-powered systems.

Adversarial ML attacks on network security are growing  

Disrupting entire networks with adversarial ML attacks is the stealth attack strategy nation-states are betting on to disrupt their adversaries’ infrastructure, which will have a cascading effect across supply chains. The 2024 Annual Threat Assessment of the U.S. Intelligence Community provides a sobering look at how important it is to protect networks from adversarial ML model attacks and why businesses need to consider better securing their private networks against adversarial ML attacks.

A recent study highlighted how the growing complexity of network environments demands more sophisticated ML techniques, creating new vulnerabilities for attackers to exploit. Researchers are seeing that the threat of adversarial attacks on ML in network security is reaching epidemic levels.

The quickly accelerating number of connected devices and the proliferation of data put enterprises into an arms race with malicious attackers, many financed by nation-states seeking to control global networks for political and financial gain. It’s no longer a question of if an organization will face an adversarial attack but when. The battle against adversarial attacks is ongoing, but organizations can gain the upper hand with the right strategies and tools.

Cisco, Cradlepoint( a subsidiary of Ericsson), DarkTrace, Fortinet, Palo Alto Networks, and other leading cybersecurity vendors have deep expertise in AI and ML to detect network threats and protect network infrastructure. Each is taking a unique approach to solving this challenge. VentureBeat’s analysis of Cisco’s and Cradlepoint’s latest developments indicates how fast vendors address this and other network and model security threats. Cisco’s recent acquisition of Robust Intelligence accentuates how important protecting ML models is to the network giant. 

Understanding adversarial attacks

Adversarial attacks exploit weaknesses in the data’s integrity and the ML model’s robustness. According to NIST’s Artificial Intelligence Risk Management Framework, these attacks introduce vulnerabilities, exposing systems to adversarial exploitation.

There are several types of adversarial attacks:

Data Poisoning: Attackers introduce malicious data into a model’s training set to degrade performance or control predictions. According to a Gartner report from 2023, nearly 30% of AI-enabled organizations, particularly those in finance and healthcare, have experienced such attacks. Backdoor attacks embed specific triggers in training data, causing models to behave incorrectly when these triggers appear in real-world inputs. A 2023 MIT study highlights the growing risk of such attacks as AI adoption grows, making defense strategies such as adversarial training increasingly important.

Evasion Attacks: These attacks alter input data to mispredict. Slight image distortions can confuse models into misclassified objects. A popular evasion method, the Fast Gradient Sign Method (FGSM) uses adversarial noise to trick models. Evasion attacks in the autonomous vehicle industry have caused safety concerns, with altered stop signs misinterpreted as yield signs. A 2019 study found that a small sticker on a stop sign misled a self-driving car into thinking it was a speed limit sign. Tencent’s Keen Security Lab used road stickers to trick a Tesla Model S’s autopilot system. These stickers steered the car into the wrong lane, showing how small carefully crafted input changes can be dangerous. Adversarial attacks on critical systems like autonomous vehicles are real-world threats.

Model Inversion: Allows adversaries to infer sensitive data from a model’s outputs, posing significant risks when trained on confidential data like health or financial records. Hackers query the model and use the responses to reverse-engineer training data. In 2023, Gartner warned, “The misuse of model inversion can lead to significant privacy violations, especially in healthcare and financial sectors, where adversaries can extract patient or customer information from AI systems.”

Model Stealing: Repeated API queries are used to replicate model functionality. These queries help the attacker create a surrogate model that behaves like the original. AI Security states, “AI models are often targeted through API queries to reverse-engineer their functionality, posing significant risks to proprietary systems, especially in sectors like finance, healthcare, and autonomous vehicles.” These attacks are increasing as AI is used more, raising concerns about IP and trade secrets in AI models.

Recognizing the weak points in your AI systems

Securing ML models against adversarial attacks requires understanding the vulnerabilities in AI systems. Key areas of focus need to include:

Data Poisoning and Bias Attacks: Attackers target AI systems by injecting biased or malicious data, compromising model integrity. Healthcare, finance, manufacturing and autonomous vehicle industries have all experienced these attacks recently. The 2024 NIST report warns that weak data governance amplifies these risks. Gartner notes that adversarial training and robust data controls can boost AI resilience by up to 30%. Implementing secure data pipelines and constant validation is essential to protecting critical models.

Model Integrity and Adversarial Training: Machine learning models can be manipulated without adversarial training. Adversarial training uses adverse examples and significantly strengthens a model’s defenses. Researchers say adversarial training improves robustness but requires longer training times and may trade accuracy for resilience. Although flawed, it is an essential defense against adversarial attacks. Researchers have also found that poor machine identity management in hybrid cloud environments increases the risk of adversarial attacks on machine learning models.

API Vulnerabilities: Model-stealing and other adversarial attacks are highly effective against public APIs and are essential for obtaining AI model outputs. Many businesses are susceptible to exploitation because they lack strong API security, as was mentioned at BlackHat 2022. Vendors, including Checkmarx and Traceable AI, are automating API discovery and ending malicious bots to mitigate these risks. API security must be strengthened to preserve the integrity of AI models and safeguard sensitive data.

Best practices for securing ML models

Implementing the following best practices can significantly reduce the risks posed by adversarial attacks:

Robust Data Management and Model Management: NIST recommends strict data sanitization and filtering to prevent data poisoning in machine learning models. Avoiding malicious data integration requires regular governance reviews of third-party data sources. ML models must also be secured by tracking model versions, monitoring production performance and implementing automated, secured updates. BlackHat 2022 researchers stressed the need for continuous monitoring and updates to secure software supply chains by protecting machine learning models. Organizations can improve AI system security and reliability through robust data and model management.

Adversarial Training: ML models are strengthened by adversarial examples created using the Fast Gradient Sign Method (FGSM). FGSM adjusts input data by small amounts to increase model errors, helping models recognize and resist attacks. According to researchers, this method can increase model resilience by 30%. Researchers write that “adversarial training is one of the most effective methods for improving model robustness against sophisticated threats.”

Homomorphic Encryption and Secure Access: When safeguarding data in machine learning, particularly in sensitive fields like healthcare and finance, homomorphic encryption provides robust protection by enabling computations on encrypted data without exposure. EY states, “Homomorphic encryption is a game-changer for sectors that require high levels of privacy, as it allows secure data processing without compromising confidentiality.” Combining this with remote browser isolation further reduces attack surfaces ensuring that managed and unmanaged devices are protected through secure access protocols.

API Security: Public-facing APIs must be secured to prevent model-stealing and protect sensitive data. BlackHat 2022 noted that cybercriminals increasingly use API vulnerabilities to breach enterprise tech stacks and software supply chains. AI-driven insights like network traffic anomaly analysis help detect vulnerabilities in real time and strengthen defenses. API security can reduce an organization’s attack surface and protect AI models from adversaries.

Regular Model Audits: Periodic audits are crucial for detecting vulnerabilities and addressing data drift in machine learning models. Regular testing for adversarial examples ensures models remain robust against evolving threats. Researchers note that “audits improve security and resilience in dynamic environments.” Gartner’s recent report on securing AI emphasizes that consistent governance reviews and monitoring data pipelines are essential for maintaining model integrity and preventing adversarial manipulation. These practices safeguard long-term security and adaptability.

Technology solutions to secure ML models

Several technologies and techniques are proving effective in defending against adversarial attacks targeting machine learning models:

Differential privacy: This technique protects sensitive data by introducing noise into model outputs without appreciably lowering accuracy. This strategy is particularly crucial for sectors like healthcare that value privacy. Differential privacy is a technique used by Microsoft and IBM among other companies to protect sensitive data in their AI systems.

AI-Powered Secure Access Service Edge (SASE): As enterprises increasingly consolidate networking and security, SASE solutions are gaining widespread adoption. Major vendors competing in this space include Cisco, Ericsson, Fortinet, Palo Alto Networks, VMware and Zscaler. These companies offer a range of capabilities to address the growing need for secure access in distributed and hybrid environments. With Gartner predicting that 80% of organizations will adopt SASE by 2025 this market is set to expand rapidly.

Ericsson distinguishes itself by integrating 5G-optimized SD-WAN and Zero Trust security, enhanced by acquiring Ericom. This combination enables Ericsson to deliver a cloud-based SASE solution tailored for hybrid workforces and IoT deployments. Its Ericsson NetCloud SASE platform has proven valuable in providing AI-powered analytics and real-time threat detection to the network edge. Their platform integrates Zero Trust Network Access (ZTNA), identity-based access control, and encrypted traffic inspection. Ericsson’s cellular intelligence and telemetry data train AI models that aim to improve troubleshooting assistance. Their AIOps can automatically detect latency, isolate it to a cellular interface, determine the root cause as a problem with the cellular signal and then recommend remediation.

Federated Learning with Homomorphic Encryption: Federated learning allows decentralized ML training without sharing raw data, protecting privacy. Computing encrypted data with homomorphic encryption ensures security throughout the process. Google, IBM, Microsoft, and Intel are developing these technologies, especially in healthcare and finance. Google and IBM use these methods to protect data during collaborative AI model training, while Intel uses hardware-accelerated encryption to secure federated learning environments. Data privacy is protected by these innovations for secure, decentralized AI.

Defending against attacks

Given the potential severity of adversarial attacks, including data poisoning, model inversion, and evasion, healthcare and finance are especially vulnerable, as these industries are favorite targets for attackers. By employing techniques including adversarial training, robust data management, and secure API practices, organizations can significantly reduce the risks posed by adversarial attacks. AI-powered SASE, built with cellular-first optimization and AI-driven intelligence has proven effective in defending against attacks on networks.