Algorithmic Warfare Is Coming. Humans Must Retain Control

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.  By Peter Maurer

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using.

Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

First Published on WEForum.org

Peter Maurer is President, International Committee of the Red Cross @ICRC

You Might Also Read: 

Killer Robots Are A Revolution in Warfare:

Robots Could Join The Ukrainian Conflict:

 

« Routers With Weak Passwords Will Soon Be Illegal In California
Cyberwars Heat Up In AsiaPac To Battle Chinese Aggression »

CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

BackupVault

BackupVault

BackupVault is a leading provider of automatic cloud backup and critical data protection against ransomware, insider attacks and hackers for businesses and organisations worldwide.

Clayden Law

Clayden Law

Clayden Law advise global businesses that buy and sell technology products and services. We are experts in information technology, data privacy and cybersecurity law.

Jooble

Jooble

Jooble is a job search aggregator operating in 71 countries worldwide. We simplify the job search process by displaying active job ads from major job boards and career sites across the internet.

IT Governance

IT Governance

IT Governance is a leading global provider of information security solutions. Download our free guide and find out how ISO 27001 can help protect your organisation's information.

ZenGRC

ZenGRC

ZenGRC (formerly Reciprocity) is a leader in the GRC SaaS landscape, offering robust and intuitive products designed to make compliance straightforward and efficient.

GrammaTech

GrammaTech

GrammaTech is a leading developer of software-assurance tools and advanced cyber-security solutions.

VivoSecurity

VivoSecurity

VivoSecurity is a pioneer in cyber risk quantification based on data science. Our products and services help organizations achieve optimal information security and GRC programs.

Sliced Tech

Sliced Tech

Sliced Tech provides enterprise grade managed Cloud services, including Security-as-a-Services, aimed at meeting the needs of commercial and government clients from within Australia.

adaware

adaware

adaware is an award-winning security and privacy software provider, empowering users to connect with confidence.

National Centre for Cyber Security (NCCS) - Pakistan

National Centre for Cyber Security (NCCS) - Pakistan

National Centre for Cyber Security (NCCS) undertakes cyber security research and plays a leading role in securing Pakistan’s Cyberspace.

eResilience

eResilience

eResilience is a division of Referentia Systems, a pioneer in an ultra-secure information safeguarding technique known as “Enclaving”, in which data can be segmented and protected within a network.

PurpleSynapz

PurpleSynapz

PurpleSynapz provides hyper-realistic Cyber Security Training with a modern curriculum and Cyber Range.

DataNumen

DataNumen

The fundamental mission of DataNumen is to recover as much data from inadvertent data disasters as possible.

Fortiphyd Logic

Fortiphyd Logic

Fortiphyd Logic equips operators of the power grid, oil & gas, and other critical infrastructure with the tools and training they need to defend their industrial networks from advanced cyberattacks.

Aurora Systems Consulting

Aurora Systems Consulting

Aurora is a Cybersecurity solutions provider with a portfolio consisting of security consulting, products and services that proactively prevent, secure and manage advanced threats and malware.

Cyber Griffin

Cyber Griffin

Founded by the City of London Police in 2017, Cyber Griffin is an initiative that supports businesses and individuals in the Square Mile to protect themselves from cyber crime.

CerraCap Ventures

CerraCap Ventures

CerraCap Ventures invest globally into early-stage B2B companies in Healthcare, Enterprise AI and Cyber Security.

Data Priva

Data Priva

Data Priva is the UK's leading subscription-based data protection, governance, risk and and compliance service.

Unified National Networks (UNN)

Unified National Networks (UNN)

UNN’s mission is to unify the national networks and create a modern and cost efficient digital platform connecting the entire country.

Tracebit

Tracebit

Tracebit uses decoys to detect and respond to cloud intrusions in minutes.

TisOva

TisOva

TisOva is an innovative cybersecurity startup dedicated to addressing the growing issue of online scams targeting students.