The Spies Of tomorrow Love Data…

Veteran intelligence officers say that, at first, many spies were deeply resistant to understanding the new dangers. But then came the lessons.

In February 2003, a CIA team was sent to Milan to conduct an "extraordinary rendition" of a suspected radical Islamist: Abu Omar was plucked off the streets and transported to Egypt.

Three years later, an Italian prosecutor using link analysis of phones, hotel reservations, car rentals and credit cards had been able to identify about two-dozen members of the CIA team and prosecute them in absentia.

What about bulk data? The fear of what could be done by using large data sets against spies was evident in Washington's neuralgic reaction to the cyber-intrusion into the federal government's Office of Personnel Management (OPM) when the personal details of 21 million government workers were stolen.

The personal details of CIA officers and other spies were not listed. That was precisely the problem - a smart intelligence service could simply correlate who at an embassy was on the OPM database and, by a process of elimination, work out that anyone not on the database was an undercover intelligence officer. In the wake of the breach, British officials were assured that there was no single database in the UK with the same amount of detail.

The moment of meeting an agent has become trickier. In the past a fleeting brush-past on a street or conversation in an alley would leave no trace unless someone had been followed. Now CCTV is everywhere and so is the data, from mobile phones and other digital tools, of where you have been. What is more, it's stored. The digital exhaust we leave behind has completely altered the ways in which spies can operate.

Countries are moving towards large biometric databases of identifiers, which can offer them knowledge about their own population. "When I joined MI6, I was trained to spot people tracking me or telephone tapping or intercepting radio communication," John Sawers, who joined MI6 in the late 70s before going to the Foreign Office and coming back as chief from 2009 to 2014, said in a speech in January 2015. "Today, those labour-intensive techniques are supported by high-end software: face recognition, footstep recognition, etc."

Sawers was brought back to MI6 in 2009 as a moderniser. That included integrating technology and the Service's "Q" team into operations much more closely. A technologist and data analyst would be brought into planning operations from the outset rather than as a last minute add-on and the case officer (who recruited the agent) became more part of a team rather than the "fighter pilot" whom everyone else served. Now the data analyst drives the operations as much as the case officer does.

Working in an age where everything is recorded and leaves a digital footprint requires different tradecraft. In some cases it means you have to, in the words of some in MI6, "go medieval" and stay offline and use old-fashioned methods of communication. Some countries were reported to have bought old typewriters in the wake of the Snowden disclosures and techniques such as secret ink are said to be making a return.

The next stage in technological transformation is coming with the growth of open-source intelligence, big data and predictive analysis. Open-source intelligence was something that spies looked down on a decade ago. Real intelligence was something that had been obtained through low cunning, not a web search.

"Open source was about routine monitoring of foreign newspapers and broadcasts for useful snippets," says Cameron Colquhoun, who worked as a government intelligence analyst before founding Neon Century, a London-based open-source intelligence company. That changed first with the Green Movement in Iran in 2009 and then with 2011's Arab Spring, which was organised, in part, on social media. "The richness of the data - geolocated, time-stamped and verifiable - meant that open source was not just something analysts could monitor but something you could use to run intelligence investigations."

One British general estimates that 85 per cent of military intelligence can now be obtained from open sources. Mapping and terrain information are simple to pick up; an understanding of local populations can be drawn using sentiment-analysis tools. So why spend huge amounts of money and take risks to get secrets when much of the information can be found? The rise of Islamic State (IS) made the importance of social media clear: British jihadists were using platforms such as Facebook to lure others in the UK to follow them.

Intelligence analysts still struggle with this world. After all, their work computers are air-gapped from the Internet, they have been encouraged not to be on social media and they normally cannot bring personal smartphones into the office. The Internet is a prime vector for espionage. Foreign spies could use it to access the systems at Vauxhall Cross. The advantages of cross-referencing information and integrating open and secret data are also huge risks because of the fear of cross-infection. Today's challenge is to leverage the Internet while not letting it into the building.

Today, analytical techniques for open data are often developed by the private sector rather than the state. The most advanced tools are being built by startups interested in sentiment analysis for commercial purposes. Just as an intelligence agency might be interested in working out who is expressing positive and influential views about a gruesome IS video, a consumer brand might be interested in social-media influencers for its product. In the US, Palantir was originally funded by In-Q-Tel, the CIA VC firm, and supports military and security programmes as well as selling its tech to consumer-facing companies.

In the UK, the startup Ripjar is moving into a similar space. "The aggregation of data is paramount to joining the dots and exposing criminal behaviour," says Tom Griffin, the company's CEO. "This is similar to the commercial world, where the true value of data is exposed when you combine the business knowledge, analytical thinking and many disparate data sets." He argues that employing techniques of machine learning and natural-language processing will not negate the need for human analysts but allows them to make sense of vast tides of data such as tweets sent by IS.

The agencies hope that big data will open the way for better intelligence analysis to avoid "strategic surprise" and provide early warning and horizon scanning. Senior CIA officials talk of their desire to build an "anticipatory intelligence capability". Sentiment analysis aims to look for early indicators of political and social crisis, unrest such as riots, signs of nascent economic instability or resource shortage. The new Alan Turing Institute, at the British Library, has formed a partnership of industry, government and academics to work on data-led solutions to various challenges, including national security.
 
But is it possible - given the volume of data and the unpredictability of human behaviour - for agencies to conduct truly insightful predictive analysis? There was an upsurge in data analysis after 9/11 when, for instance, bomb factories in Iraq were identified using patterns of phone usage by insurgents.

In the UK, GCHQ and MI6 work hand in glove. So-called bulk data is used for what is called "target discovery", finding people based on their data trails, so that more specialised techniques can be deployed. This is much harder than it used to be. In the past, a single GCHQ analyst might be able to track a dozen targets; now it can take a dozen analysts to track a single target who knows what they're doing.

This means human intelligence still plays a part. A spy inside a group such as al-Qaeda can tell you who's who and where they are even if that person practises good comms security.

Targeting individuals might be done by a close integration of human and technical intel: analysts at GCHQ might identify patters in online activity, whereas MI6 officers would try and recruit agents on the ground.

GCHQ and MI6 are moving closer together. GCHQ will sometimes need a human spy to enable an operation: think of the US-Israeli Stuxnet virus targeting Iran's nuclear programme - it needed an engineer to put a USB stick into a system. There are also pieces of info a human spy can tell you that data can't reveal. But the balance is shifting - GCHQ is roughly double the size of MI6. Inside MI6, there's an understanding that they will need a new type of spy and everyone will need digital skills.

It's becoming ever harder to keep secrets. For spies, this new world means deconstructing everything they do and analysing it for new opportunities and weaknesses, seeking out new sources of data and the latest tools to exploit. Every new trick they use to spy on someone else needs to be tested to ensure it doesn't offer an opportunity to the other side.

Nation states are working hard to exploit the insights that data offers in a new arms race of technology-driven espionage. To the victor the spoils. To the loser - as with the rest of the tech-based world, but with greater consequences - defeat and irrelevance.

Wired: http://bit.ly/1MkccXV