Algorithmic Warfare Is Coming. Humans Must Retain Control

Humanity is faced with a grave new reality – the rise of autonomous weapons. It may sound like a Hollywood script, but the risk is real: humans so far removed from wartime choices that life-and-death decision making is effectively left to sensors and software.  By Peter Maurer

What sort of future looms if the taking of a human life is relegated to algorithms? Robots and computer systems lack the distinct and unique human ability to understand both the complex environment they’re operating in and the weapons they’re using.

Moreover, the more complex weapon systems become – for example incorporating machine learning – and the greater freedom they are given to take action without human intervention, the more unpredictable the consequences in their use.

We at the International Committee of the Red Cross believe that the responsibility for decisions to kill, injure and destroy must remain with humans. It’s the human soldier or fighter – not a machine – who understands the law and the consequences of violating it, and who is responsible for applying it. These obligations cannot be transferred to a computer program. Governments – with the input of civil society and the tech industry – must waste no time in agreeing to limits on autonomy in weapon systems.

Technological advances represent great opportunities. Whether it’s in the fields of medicine, transport, agriculture, commerce, finance or virtually any other domain, robotics, AI and machine learning are having a transformative effect by advancing how we analyze and act upon data from the world around us. It makes sense that this technology be considered for national security and defense purposes, which is why it’s not a surprise that we are seeing many countries invest heavily in AI and in military robotic systems with greater autonomy.

But when it comes to armed conflict, we must not forget that even wars have limits. Governments that must now define the limits for autonomous weapon systems need to ensure compliance with international humanitarian law and be firmly rooted in the principles of humanity and the dictates of public conscience.

The good news is that when the Group of Governmental Experts charged with examining autonomous weapon systems met in April this year in Geneva, Switzerland, there was broad agreement that human control must be retained over weapon systems.

In a sense, though, that’s the easy part. At their next meeting in late August, governments must now build on this agreement to answer the more difficult question: What level of human control is required to ensure both compatibility with our laws and acceptability to our values?

The answer to this matters because some weapon systems with autonomy in their “critical functions” of selecting and attacking targets are already in use in limited circumstances for very specific tasks, such as shooting down incoming missiles or rockets. After activation by a human operator it is the weapon system that selects a target and launches a counterstrike.

However, the scope of possible future AI-powered weapon systems is much broader. It encompasses the full range of armed robotic systems, and potentially builds on the technology of AI “decision-aid” systems already being tested that analyze video feeds and detect possible targets.

With new advances happening at a breakneck pace, governments – with the support of the scientific and technical community – must take urgent action to agree on limits that will preserve the crucial human element, without stifling or slowing the technological progress that brings obvious benefits.

The alternative is the deeply unnerving prospect of a new arms race and a future where wars are fought with algorithms, with uncertain outcomes for civilians and combatants alike.

First Published on WEForum.org

Peter Maurer is President, International Committee of the Red Cross @ICRC

You Might Also Read: 

Killer Robots Are A Revolution in Warfare:

Robots Could Join The Ukrainian Conflict:

 

« Routers With Weak Passwords Will Soon Be Illegal In California
Cyberwars Heat Up In AsiaPac To Battle Chinese Aggression »

Infosecurity Europe
CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

LockLizard

LockLizard

Locklizard provides PDF DRM software that protects PDF documents from unauthorized access and misuse. Share and sell documents securely - prevent document leakage, sharing and piracy.

CSI Consulting Services

CSI Consulting Services

Get Advice From The Experts: * Training * Penetration Testing * Data Governance * GDPR Compliance. Connecting you to the best in the business.

ManageEngine

ManageEngine

As the IT management division of Zoho Corporation, ManageEngine prioritizes flexible solutions that work for all businesses, regardless of size or budget.

IT Governance

IT Governance

IT Governance is a leading global provider of information security solutions. Download our free guide and find out how ISO 27001 can help protect your organisation's information.

Authentic8

Authentic8

Authentic8 transforms how organizations secure and control the use of the web with Silo, its patented cloud browser.

Blue Frost Security

Blue Frost Security

Blue Frost Security provides high-level IT security consulting, penetration testing services, ISO 27001 Solutions, PCI compliance solutions and training.

National Cyber Directorate Israel

National Cyber Directorate Israel

The Israeli National Cyber Directorate provides incident handling services for civilian entities and critical infrastructures and works to increase national resilience against cyber threats.

Auxilium Cyber Security

Auxilium Cyber Security

Auxilium Cyber Security is independent information security consultancy company providing cyber security services tailored to meet the evolving needs of organizations worldwide.

Ponemon Institute

Ponemon Institute

Ponemon Institute conducts independent research on data protection and emerging information technologies.

Flipside

Flipside

Information Security training provider specialized in personalized training and security awareness campaigns.

Slovak National Accreditation Service (SNAS)

Slovak National Accreditation Service (SNAS)

SNAS is the national accreditation body for Slovakia. The directory of members provides details of organisations offering certification services for ISO 27001.

CyberSwarm

CyberSwarm

CyberSwarm is developing a neuromorphic System-on-a-Chip dedicated to cybersecurity which helps organizations secure communication between connected devices and protect critical business assets.

Aries Security

Aries Security

Aries Security provides a premiere cyber training range and skills assessment suite and develops content for all levels of ability.

MAXXeGUARD Data Safety

MAXXeGUARD Data Safety

MAXXeGUARD: The High Security Shredder. MAXXeGUARD easily destroys hard disks up to the highest security levels as well as other digital data carriers like SSD’s, LTO’s, USB’s, CD’s etc.

Sansec

Sansec

Sansec is the global leader in eCommerce malware and vulnerability detection. We help you to stay ahead of hackers!

Plex IT

Plex IT

Plex IT provides managed IT services to organisations along with managed security services.

Ignite Cyber

Ignite Cyber

IGNITE Cyber is focused on enabling secure technology adoption through intelligent business decisions. We are focused on providing a secure and stable business environment for everyone.

Vana Solutions

Vana Solutions

Vana Solutions is an Information Technology Services company. We help commercial & federal organizations select, adapt, and integrate the right technology solution so you can move faster.

ITButler e-Services

ITButler e-Services

At IT Butler, our mission is crystal clear: we are dedicated to providing top-tier cybersecurity solutions and best-practice methodologies to secure and enhance your digital infrastructure’s resilienc

CrashPlan

CrashPlan

CrashPlan provides peace of mind through secure, scalable, and straightforward endpoint data backup.

DefectDojo

DefectDojo

DefectDojo is a DevSecOps and vulnerability management tool.