The AI Apocalypse

Adversarial attacks are an increasingly worrisome threat to the performance of artificial intelligence applications.

If an attacker can introduce nearly invisible alterations to image, video, speech, and other data for the purpose of fooling AI-powered classification tools, it will be difficult to trust this otherwise sophisticated technology to do its job effectively.

Imagine how such attacks could undermine AI-powered autonomous vehicles ability to recognise obstacles, content filters’ effectiveness in blocking disturbing images, or in access systems’ ability to deter unauthorized entry.

Some people argue that adversarial threats stem from “deep flaws” in the neural net technology that powers today’s AI. After all, it’s well-understood that many machine learning algorithms are vulnerable to adversarial attacks.

However, you could just as easily argue that this problem calls attention to weaknesses in enterprise processes for building, training, deploying, and evaluating AI models.

None of these issues are news to AI experts. It’s true that the AI community lacks any clear consensus on best practices for building anti-adversarial defenses into deep neural networks. But from what I see in the research literature and industry discussions, the core approaches from which such a framework will emerge are already crystallising.

Going forward, AI developers will need to follow these guidelines to build anti-adversarial protections into their applications:

Assume attacks on all in-production AI assets

As AI is deployed everywhere, developers need to assume that their applications will be high-profile sitting ducks for adversarial manipulation.

AI exists to automate cognition, perception, and other behaviors that, if they produce desirable results, might merit the praise one normally associates with “intelligence.” However, AI’s adversarial vulnerabilities might result in cognition, perception, and other behaviors of startling stupidity, perhaps far worse than any normal human being would have exhibited under the circumstances.

Perform adversarial risks prior to AI development

Upfront and throughout the life cycle of their AI apps, developers should frankly assess their projects’ vulnerability to adversarial attacks.

As noted in a 2015 research paper published by the IEEE, developers should weigh the possibility of unauthorized parties gaining direct access to key elements of the AI project, including the neural net architecture, training data, hyper-parameters, learning methodology, and loss function being used.

Alternatively, the paper shows, an attacker might be able to collect a surrogate dataset from the same source or distribution as the training data used to optimize an AI neural net model. This could provide the adversary with insights into what type of ersatz input data might fool a classifier model that was built with the targeted deep neural net.

In another attack approach described by the paper, even when the adversary lacks direct visibility into the targeted neural net and associated training data, attackers could exploit tactics that let them observe “the relationship between changes in inputs and outputs … to adaptively craft adversarial samples.”

Generate AI adversarial examples

AI developers should immerse themselves in the growing body of research on the many ways in which subtle adversarial alterations may be introduced into the images processed by convolutional neural networks (CNNs).

Data scientists should avail themselves of the growing range of open source tools, such as this one on GitHub, for generating adversarial examples to test the vulnerability of CNNs and other AI models. More broadly, developers should consider the growing body of basic research that focuses on generating adversarial examples for training generative adversarial networks(GANs) of all sorts, including those that aren’t directly focused on fending off cybersecurity attacks.

Rely on human curators and algorithmic discriminators

The effectiveness of an adversarial attack depends on its ability to fool your AI apps’ last line of defense.

Adversarial manipulation of an image might be obvious to the naked eye but still somehow fool a CNN into misclassifying it. Conversely, a different manipulation might be too subtle for a human curator to detect, but a well-trained discriminator algorithm in GAN may be able to pick it out without difficulty.

One promising approach to second issue is to have a GAN in which an adversary model alters each data point in an input image, thereby trying to maximize classification errors, while a countervailing discriminator model tries to minimise misclassification errors.

Build AI algorithms for detecting adversarial examples

Some algorithms may be more sensitive than others to the presence of adversary-tampered images and other data objects. For example, researchers at the University of Campinas found a scenario in which a shallow classifier algorithm might detect adversarial images better than a deeper-layered CNN. They also found that some algorithms are best suited for detecting manipulations across an entire image, while others may be better at finding subtle fabrications in one small section of an image.

One approach for immunizing CNNs from these attacks might be to add what Cornell University researcher Arild Nøkland calls an “adversarial gradient” to the backpropagation of weights during an AI model’s training process. It would be prudent for data science teams to test the relative adversary-detection advantages of different algorithms using ongoing A/B testing both in development and production environments.

Re-use adversarial-defense knowledge

As noted in a 2016 research paper published by the IEEE, data scientists can use transfer-learning techniques to reduce the sensitivity of a CNN or other model to adversarial alterations in input images.

Whereas traditional transfer learning involves applying statistical knowledge from an existing model to a different one, the paper discusses how a model’s existing knowledge, gained through training on a valid data set, might be “distilled” to spot adversarial alterations.

According to the authors, “we use defensive distillation to smooth the model learned by a [distributed neural net] architecture during training by helping the model generalise better to samples outside of its training dataset.”

The result is that a model should be better able to recognise the difference between adversarial examples (those that resemble examples in its training set) and non-adversarial examples (those that may deviate significantly from those in its training set).

Without these practices as a standard part of their methodology, data scientists might inadvertently bake automated algorithmic gullibility into their neural networks. As our lives increasingly rely on AI to do the smart thing in all circumstances, these adversarial vulnerabilities might prove catastrophic.

That’s why it’s essential that data scientists and AI developers put in place suitable safeguards to govern how AI apps are developed, training, and managed.

InfoWorld

You Might Also Read: 

Artificial Intelligence: A Warning:

What Happens If Criminals & Terrorists Get To Use AI?:

 

« Firms Underrate The ‘Seismic Aftershock’ Of An Attack
A Robot Won’t Steal Your Job Just Yet… »

CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

LockLizard

LockLizard

Locklizard provides PDF DRM software that protects PDF documents from unauthorized access and misuse. Share and sell documents securely - prevent document leakage, sharing and piracy.

Perimeter 81 / How to Select the Right ZTNA Solution

Perimeter 81 / How to Select the Right ZTNA Solution

Gartner insights into How to Select the Right ZTNA offering. Download this FREE report for a limited time only.

Jooble

Jooble

Jooble is a job search aggregator operating in 71 countries worldwide. We simplify the job search process by displaying active job ads from major job boards and career sites across the internet.

NordLayer

NordLayer

NordLayer is an adaptive network access security solution for modern businesses — from the world’s most trusted cybersecurity brand, Nord Security. 

North Infosec Testing (North IT)

North Infosec Testing (North IT)

North IT (North Infosec Testing) are an award-winning provider of web, software, and application penetration testing.

SecureAuth

SecureAuth

SecureAuth delivers cutting edge identity and information security solutions for cloud, mobile, web, and VPN systems.

Cypress Semiconductor

Cypress Semiconductor

Cypress is a semiconductor design and manufacturing company providing embedded devices for secure IoT applications.

Apcon

Apcon

Apcon's mission is to provide valuable network insights that enable security and network professionals to monitor, secure and protect their data in both physical and virtual environments.

NXP Semiconductors

NXP Semiconductors

NXP is a world leader in secure connectivity solutions for embedded applications and the Internet of Things.

LEXFO

LEXFO

LEXFO specializes in the security of information systems, assisting clients in protecting information assets using an offensive and innovative approach.

Vicarious

Vicarious

Vicarious identify the vulnerabilities in the software without involving the vendor and protect it before the hackers take advantage of it, even in compiled applications.

Avatao

Avatao

Avatao is an online training platform for building secure software, offering a rich library of hands-on IT security exercises for software engineers to teach secure programming.

Ekran System

Ekran System

Ekran System is an advanced insider threat detection solution for companies of any size.

Secure Recruitment

Secure Recruitment

Secure Recruitment is a specialist Executive Search business that focuses its efforts on attracting specific exceptional talent in Cyber Security.

Citalid

Citalid

The Citalid cyber risk management platform combines threat and business intelligence to identify the risks scenarios you face.

Simplilearn

Simplilearn

Simplilearn is the world's #1 online bootcamp for digital skills training in disciplines such as Cyber Security, Cloud Computing, Project Management, Digital Marketing, and Data Science.

Laminar

Laminar

Laminar provides the only Public Cloud Data Protection solution that provides full visibility and enforcement capabilities across your entire public cloud infrastructure.

Cardonet

Cardonet

Cardonet is an IT Support and IT Services business offering end-to-end IT services, 24x7 IT Support to IT Consultancy, Managed IT and Cyber Security.

Xscale Accelerator

Xscale Accelerator

Xscale's vision is to create world-class startups out of India by transforming sales and providing access to global markets.

DarkFeed

DarkFeed

DarkFeed is a Threat Intelligence provider that monitors the darknet in real-time, where hackers and Cyber criminals are most active.

IONIX

IONIX

IONIX is the attack surface management solution that uses Connective Intelligence to shine a spotlight on exploitable risks across your real attack surface and its digital supply chain.