Algorithmic Warfare Is Coming. Humans Must Retain Control

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.  By Peter Maurer

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using.

Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

First Published on WEForum.org

Peter Maurer is President, International Committee of the Red Cross @ICRC

You Might Also Read: 

Killer Robots Are A Revolution in Warfare:

Robots Could Join The Ukrainian Conflict:

 

« Routers With Weak Passwords Will Soon Be Illegal In California
Cyberwars Heat Up In AsiaPac To Battle Chinese Aggression »

CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

Syxsense

Syxsense

Syxsense brings together endpoint management and security for greater efficiency and collaboration between IT management and security teams.

Clayden Law

Clayden Law

Clayden Law advise global businesses that buy and sell technology products and services. We are experts in information technology, data privacy and cybersecurity law.

The PC Support Group

The PC Support Group

A partnership with The PC Support Group delivers improved productivity, reduced costs and protects your business through exceptional IT, telecoms and cybersecurity services.

MIRACL

MIRACL

MIRACL provides the world’s only single step Multi-Factor Authentication (MFA) which can replace passwords on 100% of mobiles, desktops or even Smart TVs.

NordLayer

NordLayer

NordLayer is an adaptive network access security solution for modern businesses — from the world’s most trusted cybersecurity brand, Nord Security. 

Apomatix

Apomatix

Apomatix is a platform that simplifies the complexity of cyber risk audit and management.

Hedgehog Security

Hedgehog Security

The key objective of Hedgehog is to provide simple, effective and affordable information security improvements that support your drive to increase productivity and profitability.

Nexusguard

Nexusguard

Nexusguard is at the forefront of the fight against malicious Internet attacks, protecting organizations worldwide from threats to their websites, services, and reputations.

Securely

Securely

Securely Ltd. is an IT consulting and services firm specializing in PKI solutions and products.

The Media Trust

The Media Trust

The Media Trust continuously scans websites, ad tags and mobile apps and alerts on anomalies affecting websites and visitors.

London Office for Rapid Cybersecurity Advancement (LORCA)

London Office for Rapid Cybersecurity Advancement (LORCA)

LORCA's mission is to support the most promising cyber security innovators in growing solutions to meet the most pressing industry challenges and build the UK’s international cyber security profile.

Network Integrated Business Solutions (NIBS)

Network Integrated Business Solutions (NIBS)

NIBS is an IT services provider offering a range of services with the aim of simplifying and securing technology.

Consortium for Information & Software Quality (CISQ)

Consortium for Information & Software Quality (CISQ)

The mission of CISQ is to develop international standards for software quality and to promote the development and sustainment of secure, reliable, and trustworthy software.

Specops Software

Specops Software

Specops Software is a leading password management and authentication solution vendor.

Field Effect Software

Field Effect Software

Field Effect Software build sophisticated and integrated IT security, threat surface reduction, training and simulation capabilities for enterprises and small businesses.

Nineteen Group

Nineteen Group

Nineteen Group delivers major-scale exhibitions within the security, fire, emergency services, health and safety, facilities management and maintenance engineering sectors.

Sollensys

Sollensys

Sollensys is a leader in commercial blockchain applications. Our flagship product, The Blockchain Archive Server™ is the best defense against the devastating financial loss that ransomware causes.

Mitnick Security

Mitnick Security

Mitnick Security is a leading global provider of information security consulting and training services.

HEQA Security

HEQA Security

HEQA Security (formerly QuantLR) offer the world’s most cost-effective, easy-to-integrate, and secure Quantum Key Distribution (QKD) solution

Proximus Ada

Proximus Ada

Proximus Ada is the first Belgian center of excellence combining artificial intelligence and cybersecurity.

SkillsDA

SkillsDA

SkillsDA is pureplay company in cyber security involved in capacity building towards National Security.