Social Media Companies Scan For Potential Terrorists

Following the politically motivated shooting in Pittsburgh and the mailing of pipe bombs to political officials and journalists across the country, public outcry has risen against social-media companies. 

Suspected pipe-bomber Cesar Sayoc and Pittsburgh shooting suspect Robert Bowers used various platforms to post content indicating their potential for ideological violence. 

Some have asked why social-media companies didn’t do more, sooner, to stop the threat.

It’s a question that Facebook, Twitter, YouTube, and others have faced before, going back to 2014 when the problem was content from extremists of a different sort: violent jihadist groups such as ISIS.

Since then, many social-media giants have developed technological and policy-based ways to help prevent extremist content from proliferating across their sites and even to help law enforcement better track potential violent actors. But those efforts were aimed at foreign Islamic extremists, not domestic threats.

Monika Bickert, Facebook’s head of product policy and counterterrorism, has helped her company move farther along in this regard than some others. 

What role did Facebook play in the events that unfolded last week? A limited one: Robert Bowers, the charged Pittsburgh shooter, kept his threats and violent posts to a relatively obscure right-wing platform called Gab. Soon after he joined Gab in January, he began to post and spread images and content threatening to Jews.

Cesar Sayoc had a Facebook profile that he used to advance conspiracy theories. He also threatened people on Twitter, such as political analyst Rochelle Ritchie. Ritchie reported the threats to the platform. Last weekend, the company apologized for failing to act sooner.

Although Sayoc had a small presence on Facebook, the company might still have had a lot of information about him. Facebook monitors extremist rhetoric and content, of the Islamist variety, on sites that aren’t Facebook. It employs contractors to watch extremist chat rooms and other places so they can be ready to identify and tag threatening language, images, and content on Facebook.

Facebook’s Erin Marie Saltman, a policy manager at Facebook who oversees counterterrorism efforts in Europe, Africa, and the Middle East, disclosed this at the GLOBSEC security summit in May.

“There are a lot of people in other parts of the world that are not Facebook and not government,” Saltman said. “They are intel providers that sit and squat on a lot of these other sites and they tell us, in as close to real time as possible, when bad content is being released and so we know about it as soon as possible. 

So when the Abu Bakr al-Baghdadi speech was released a little while ago, and it wasn’t in video form, just audio, we were able to hash[tag] it before it started hitting our site.”

Predicting potentially violent behavior requires as much digitally collected data as possible, precisely the sort of data that intel vendors watching sites like Gab might notice. But when Defense One asked Facebook representatives whether they monitor sites like Gab for such content, or potential indicators of violence, they declined to say.

“As Erin mentioned, we work with intel and research firms who monitor many platforms, but we prefer not to disclose further details as bad actors actively work to circumvent our detection techniques,” a Facebook spokesperson said. 

“Since the bombing attempts, and the shooting in Pittsburgh, teams across our company have been monitoring developments in real time to understand both situations and how they relate to content on our site,” they added.

In 2011, J. Reid Meloy, a forensic psychologist and consultant to FBI’s Behavioral Analysis Units at Quantico, identified eight behaviors that can predict lone-wolf attacks based on ideological extremism. Sayoc and Bowers exhibited several of them across multiple social media sites. 

If social-media companies could search for these subtle indicators of a potentially dangerous person, behaviors, such as fixation or obsession, in the context of overtly troubling posts and comments such as direct threats, patterns could emerge to predict an individual’s behavior.

Cross-platform analysis of individuals’ data residue is what contemporary micro-targeting for advertisements is based on. It works to predict whether a person might be open to a specific product pitch but it also works to predict potentially harmful behavior. 

Facebook is already using AI to spot suicidal tendencies signaled by text patterns. The same algorithms could be applied to spot violent extremism, as could network analysis and even semantic text analysis. 

That information, coupled with the identification of violent messages or threats spread on other sites, could go a long way toward predicting and preventing violent behavior and the posting of extremist content. And it is, but it’s mostly violent Islamic behavior and content.

Consider the case of Demetrius Nathaniel Pitts, a Cleveland man recently charged with plotting a jihadist-inspired terrorist attack. Authorities monitored Pitts’s Facebook posts carefully after he commented on a photo of an al-Qaida training camp. 

His posts extorted Muslims to learn how to operate firearms, posts that law enforcement officials described as “disturbing” to USA Today. But pages urging non-Muslims (or people who are not explicitly Muslim) to own and practice with firearms are common on Facebook.

“We continually enforce our Community Standards through a combination of technology, reports from our community, and human review. This includes our hate speech policy that prohibits content that attacks people based on their race, ethnicity, national origin, religious affiliations, sexual orientation, caste, sex, gender, gender identity, and serious disease or disability,” said the spokesperson.

Following the public outcry against the proliferation of jihadist extremist messaging, Facebook and other sites tried a technique called hashing: essentially, to mark Islamic extremist content as individuals tried to spread it from one site to another. In 2016, Facebook executives led an effort to share data on hashed images across platforms.

“It creates the equivalent of a digital fingerprint so you can know when these things are coming up. We encourage that type of sharing, the hash sharing. Anybody using types of video, photo matching, would be able to use the hashes we are trying to share,” said Saltman.

Could hashed images and data from accounts like Bowers’s and Sayoc’s be relevant to law enforcement? Potentially, but the practice of hash sharing doesn’t involve the government, said Saltman. Instead, she said, the goal was to make a “safe tech space” for technology platforms to use whatever tools they saw fit.

“This is a by-industry, for-industry effort; it doesn’t include government or NGOs. It’s really so we can create a safe space so that some of these smaller platforms that are really scared about talking outside of industry, and admitting you have a problem is step one, can come together in a safe tech space and start operationalising around some of this.”

In a conversation with New York Times reporters on Sunday, Gab founder Andrew Torba denied that he or any Gab employer should monitor content on the site. 

“Twitter and other platforms police ‘hate speech’ as long as it isn’t against President Trump, white people, Christians, or minorities who have walked away from the Democratic Party,” he wrote. “This double standard does not exist on Gab.”

Defense One:

You Might Also Read:

A Genocide Incited On Facebook:

The Weaponization Of Social Media

« British Policing Faces The Future
Blockchain Can Improve Manufacturing »

Infosecurity Europe
CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

CSI Consulting Services

CSI Consulting Services

Get Advice From The Experts: * Training * Penetration Testing * Data Governance * GDPR Compliance. Connecting you to the best in the business.

ZenGRC

ZenGRC

ZenGRC (formerly Reciprocity) is a leader in the GRC SaaS landscape, offering robust and intuitive products designed to make compliance straightforward and efficient.

The PC Support Group

The PC Support Group

A partnership with The PC Support Group delivers improved productivity, reduced costs and protects your business through exceptional IT, telecoms and cybersecurity services.

Practice Labs

Practice Labs

Practice Labs is an IT competency hub, where live-lab environments give access to real equipment for hands-on practice of essential cybersecurity skills.

Resecurity

Resecurity

Resecurity is a cybersecurity company that delivers a unified platform for endpoint protection, risk management, and cyber threat intelligence.

CEPS

CEPS

CEPS is a leading think tank and forum for debate on EU affairs, ranking among the top think tanks in Europe. Topic areas include Innovation, Digital economy and Cyber-security.

HyTrust

HyTrust

HyTrust specialises in security, compliance and control software for virtualization and cloud environments.

Cyber London (CyLon)

Cyber London (CyLon)

CyLon is a leading cyber security accelerator and seed investment programme. We help entrepreneurs from across the globe to build cyber security businesses, raise investment, and develop partnerships.

ISARA Corp

ISARA Corp

ISARA Corporation is a security solutions company specializing in creating class-defining quantum-safe cryptography for today's computing ecosystems.

National Cybersecurity Institute (NCI) - Excelsior College

National Cybersecurity Institute (NCI) - Excelsior College

NCI is Excelsior College’s research center dedicated to assisting government, industry, military and academic sectors meet the challenges in cybersecurity policy, technology and education.

Miratech

Miratech

Miratech is a global IT services and consulting organization offering a full range of IT infrastructure solutions and services including cyber security.

Nucleon

Nucleon

Nucleon enables cybersecurity tools, organizations and software developers to become proactive by blocking threats before they become breaches.

Deepnet Security

Deepnet Security

Deepnet Security is a leading security software developer and hardware provider in Multi-Factor Authentication (MFA), Single Sign-On (SSO) and Identity & Access Management (IAM).

Securd

Securd

Securd takes opportunities away from your cyber adversaries. Cloud-delivered zero-trust DNS firewall and web filtering protection keep your business network and remote employees safe.

Appsian Security

Appsian Security

Appsian provides powerful solutions that help organizations take control of their business critical data and financial transactions.

CV-Library

CV-Library

Start your job search with 216,931 live UK vacancies on award-winning CV-Library. Register your CV and find local jobs near you today!

Emerge Digital

Emerge Digital

Emerge Digital is a technology and digital innovation business and Managed Services Provider providing solutions to SMEs.

Knownsec

Knownsec

Knownsec provides customers with cloud defense, cloud monitoring, and cloud mapping products and services with "AI + security big data" as the underlying capability.

SecureFlag

SecureFlag

SecureFlag is dedicated to enhancing secure coding across all technical profiles within the Software Development Lifecycle.

Attura

Attura

Atturra is one of Australia's leading advisory and IT solutions providers, focused on providing end-to-end transformation services to its clients.

Servadus

Servadus

Servadus help organizations with their cybersecurity and compliance programs through management and sustainability, consulting, and assessing.