Combating The Threat Of Malicious AI

A group of academics and researchers from leading universities and think-tanks, including Oxford, Yale, Cambridge and Open AI, recently published a chilling report titled, The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation. 
 
The report raised alarm bells about the rising possibilities that rogue states, criminals, terrorists and other malefactors could soon exploit AI capabilities to cause wide spread harm.
 
These risks are weighty and disturbing, albeit not surprising. Several politicians and humanitarians have repeatedly advocated for the need to regulate AI, with some calling it humanity’s most plausible existential threat.
 
For instance, back in 2016, Barack Obama, then President of the United States, publicly admitted his fears that an AI algorithm could be unleashed against US nuclear weapons. “There could be an algorithm that said, ‘Go penetrate the nuclear codes and figure out how to launch some missiles,'” Obama cautioned.  
 
A year later, in August 2017, the charismatic Tesla and SpaceX CEO, Elon Musk, teamed up with 116 executives and scholars to sign an open letter to the UN, urging the world governing body to urgently enact statutes to ban the global use of lethal autonomous weapons or so-called “killer robots.”
 
While AI’s ability to boost fraud detection and cyber defense is unquestionable, this vital role could soon prove to be a zero-sum game. 
 
The same technology could be exploited by malefactors to develop superior and elusive AI programs that will unleash advanced persistent threats against critical systems, manipulate stock markets, perpetrate high-value fraud or steal intellectual property. 
 
What makes this new report particularly significant is its emphasis on the immediacy of the threat. It predicts that widespread use of AI for malicious purposes, such as repurposed autonomous weapons, automated hacking, target impersonation, highly tuned phishing attacks, etc., could all eventuate as early as the next decade.
 
So, why has this malicious AI threat escalated from Hollywood fantasy to potential reality far more rapidly than many pundits anticipated? 
 
There are three primary drivers: 
  • First, cyber-threat actors are increasingly agile and inventive, spurred by the growing base of financial resources and absence of regulation, factors that often stifle innovation for legitimate enterprises.
  • Secondly and perhaps most important, the rapid intersection between cyber-crime and politics, combined with deep suspicions that adversarial nations are using advanced programs to manipulate elections, spy on military programs or debilitate critical infrastructure, have further dented prospects of meaningful international cooperation. 
  • Thirdly, advanced AI-based programs developed by nation-states may inadvertently fall into wrong hands. 
An unsettling example is the 2016 incident, in which a ghostly group of hackers, going by the moniker “The Shadow Brokers,” reportedly infiltrated the US National Security Agency (NSA) and stole advanced cyber weapons that were allegedly used to unleash the WannaCry ransomware in May 2017. 
 
As these weapons become more powerful and autonomous, the associated risks will invariably grow. The prospect of an autonomous drone equipped with hellfire missiles falling into wrong hands, for instance, would be disconcerting to us all. 
It’s clear that addressing this grave threat will be complex and pricey, but the task is pressing. As report co-author Dr. Seán Ó hÉigeartaigh stressed, “We live in a world that could become fraught with day-to-day hazards from the misuse of AI and we need to take ownership of the problems, because the risks are real.” Several strategic measures are required, but the following two are urgent: 
  • There is need for deeper, transparent and well-intentioned collaboration between academics, professional associations, the private sector, regulators and world governing bodies. This threat transcends the periphery of any single enterprise or nation. Strategic collaboration will be more impactful than unilateral responses. 
  • As the report highlighted, we can learn from disciplines such as cybersecurity that have a credible history in developing best practices to handle dual-use risks. 
Again, while this is an important step, much more is required. As Musk and his co-collaborators wrote to the UN, addressing this risk requires binding international laws. After all, regulations and standards are only as good as their enforcement.  
This is an old story; history is repeating itself. As Craig Timberg wrote in The Threatened Net: How the Web Became a Perilous Place, “When they [Internet designers] thought about security, they foresaw the need to protect the network against potential intruders and military threats, but they didn’t anticipate that the Internet’s own users would someday use the Internet to attack one another.”
 
The Internet’s rapid transformation from a safe collaboration tool to a dangerous place provides an important lesson. If we discount this adjacent threat, AI’s capabilities, which hold so much promise, will similarly be exploited by those with bad intentions.  
 
Absent a coherent international response, the same technology that is being used to derive deep customer insights, tackle complex and chronic ailments, alleviate poverty and advance human development could be misappropriated and lead to grave consequences.
 
ISACA
 
You Might Also Read: 
 
Artificial Intelligence: A Warning:
 
Artificial Intelligence, Robotics & All Tomorrows Wars:
 
« Artificial Intelligence Is Cyber Defence
N Korea Is A Bigger Cyber Threat Than Russia »

Infosecurity Europe
CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

Infosecurity Europe, 3-5 June 2025, ExCel London

Infosecurity Europe, 3-5 June 2025, ExCel London

This year, Infosecurity Europe marks 30 years of bringing the global cybersecurity community together to further our joint mission of Building a Safer Cyber World.

Practice Labs

Practice Labs

Practice Labs is an IT competency hub, where live-lab environments give access to real equipment for hands-on practice of essential cybersecurity skills.

LockLizard

LockLizard

Locklizard provides PDF DRM software that protects PDF documents from unauthorized access and misuse. Share and sell documents securely - prevent document leakage, sharing and piracy.

Syxsense

Syxsense

Syxsense brings together endpoint management and security for greater efficiency and collaboration between IT management and security teams.

The PC Support Group

The PC Support Group

A partnership with The PC Support Group delivers improved productivity, reduced costs and protects your business through exceptional IT, telecoms and cybersecurity services.

Willis Towers Watson

Willis Towers Watson

Willis Towers Watson is a global risk management, insurance brokerage and advisory company. Services offered include Cyber Risks insurance.

ClickDatos

ClickDatos

ClickDatos specializes in consulting, auditing, data protection training, accredited by ISO/IEC 27001 certification.

Apomatix

Apomatix

Apomatix is a platform that simplifies the complexity of cyber risk audit and management.

Mondo

Mondo

Mondo is the largest national staffing agency specializing exclusively in high-end, niche IT, Tech, and Digital Marketing talent. Areas of expertise include Cybersecurity.

NopSec

NopSec

NopSec provides automated IT security control measurement and risk remediation solutions to help businesses protect their IT environments from security breaches.

CYE

CYE

Utilizing data, numbers, and facts, CYE helps security leaders know what business assets are at risk and execute cost-effective remediation projects for optimal risk prevention.

Security Engineered Machinery (SEM)

Security Engineered Machinery (SEM)

SEM provides comprehensive end-of-life solutions for the protection of sensitive information in government and commercial markets.

The Cyber AB

The Cyber AB

The Cyber AB is the official accreditation body of the Cybersecurity Maturity Model Certification (CMMC) Ecosystem.

TM One

TM One

TM One is the enterprise and public sector business solutions arm of Telekom Malaysia Berhad (TM) Group.

C3.ai Digital Transformation Institute

C3.ai Digital Transformation Institute

The C3.ai Digital Transformation Institute is a research consortium dedicated to accelerating the benefits of artificial intelligence for business, government, and society.

Anonomatic

Anonomatic

Anonomatic’s mission is to make data privacy secure, simple and cost effective. We are Data and Privacy Experts who are passionate about helping organizations solve PII compliance.

Symbol Security

Symbol Security

Through situational learning, simulations, and a gamified user experience, Symbol strengthens the cyber awareness of employees and helps companies lower cyber risk.

Patriot Consulting Technology Group

Patriot Consulting Technology Group

Patriot Consulting's mission is to help our clients manage cybersecurity risk through secure deployments of Microsoft 365.

One82

One82

Serving emerging small and medium-sized businesses in California and neighboring regions for over 20 years, One82 has established itself as the most dependable provider of IT support services.

Nuance Communications

Nuance Communications

From revolutionizing the doctor-patient relationship to reinventing the way brands connect with their customers, Nuance technology helps organizations push the boundaries of what’s possible.

Brunswick Group

Brunswick Group

Brunswick is a critical issues firm. We advise the world’s leading companies on how to navigate the critical issues they face and engage with their critical stakeholders.