Algorithmic Warfare Is Coming. Humans Must Retain Control

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.  By Peter Maurer

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using.

Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

First Published on WEForum.org

Peter Maurer is President, International Committee of the Red Cross @ICRC

You Might Also Read: 

Killer Robots Are A Revolution in Warfare:

Robots Could Join The Ukrainian Conflict:

 

« Routers With Weak Passwords Will Soon Be Illegal In California
Cyberwars Heat Up In AsiaPac To Battle Chinese Aggression »

CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

Clayden Law

Clayden Law

Clayden Law advise global businesses that buy and sell technology products and services. We are experts in information technology, data privacy and cybersecurity law.

IT Governance

IT Governance

IT Governance is a leading global provider of information security solutions. Download our free guide and find out how ISO 27001 can help protect your organisation's information.

Authentic8

Authentic8

Authentic8 transforms how organizations secure and control the use of the web with Silo, its patented cloud browser.

ZenGRC

ZenGRC

ZenGRC - the first, easy-to-use, enterprise-grade information security solution for compliance and risk management - offers businesses efficient control tracking, testing, and enforcement.

ON-DEMAND WEBINAR: What Is A Next-Generation Firewall (and why does it matter)?

ON-DEMAND WEBINAR: What Is A Next-Generation Firewall (and why does it matter)?

Watch this webinar to hear security experts from Amazon Web Services (AWS) and SANS break down the myths and realities of what an NGFW is, how to use one, and what it can do for your security posture.

F-Secure

F-Secure

F-Secure defends enterprises and consumers against everything from opportunistic ransomware infections to advanced cyber attacks.

Phoenix Contact Cyber Security

Phoenix Contact Cyber Security

Phoenix Contact Cyber Security is a leading manufacturer of network security appliances for use in industrial environments.

CyberGRX

CyberGRX

The CyberGRX Exchange and our risk assessments-as-a-service help Enterprises and Third Parties cost-effectively identify, prioritize and mitigate risk.

Kippeo Technologies

Kippeo Technologies

Kippeo is a security systems integrator providing innovative solutions that look at all the parameters and connect all the dots.

Swedish Board for Accreditation and Conformity Assessment (SWEDAC)

Swedish Board for Accreditation and Conformity Assessment (SWEDAC)

SWEDAC is the national accreditation body for Sweden. The directory of members provides details of organisations offering certification services for ISO 27001.

Robert Walters

Robert Walters

Robert Walters is one of the world's leading global specialist professional recruitment and recruitment process outsourcing consultancies.

GateKeeper Enterprise

GateKeeper Enterprise

The GateKeeper Enterprise software is an identity access management solution. Automated proximity-based authentication into computers and websites. Passwordless login and auto-lock PCs.

MazeBolt Technologies

MazeBolt Technologies

Israel-based MazeBolt is an innovation leader in cybersecurity, with over two decades of experience in pioneering DDoS protection solutions.

Dazz

Dazz

Dazz is the cloud security remediation platform for smart security and development teams.

Ostra Cybersecurity

Ostra Cybersecurity

As a next-generation MSSP, Ostra Cybersecurity combines best-in-class tools, proprietary technology and exceptional talent to deliver Fortune 100-level protection for businesses of all sizes.

Apollo Information Systems

Apollo Information Systems

Apollo is a value-added reseller that provides our clients with the complete set of cybersecurity and networking services and solutions.

Gen Digital

Gen Digital

At Gen™, our mission is to create technology solutions for people to take full advantage of the digital world, safely, privately, and confidently – so together, we can build a better tomorrow.

Quantum Ventura

Quantum Ventura

Quantum Ventura is a technology innovation company with a single mission of delivering customer-centric advanced solutions to US Federal & State Governments and Private Sector customers.

DHCO IT

DHCO IT

The DHCO IT team are experts in IT support, cyber security, cloud support and disaster recovery, and are Microsoft 365 partners.

Cyber Industrial Networks

Cyber Industrial Networks

Cyber Industrial Networks objective is to service the needs of industry in achieving reliable, robust and secure infrastructure that supports productivity.

Acumen

Acumen

Acumen's cyber security engineers protect your critical systems, in critical moments. We are here when you need us most.