Decrypting the Dark Web
Data analysis to be presented at Black Hat Europe highlights trends in communication between bad actors who gather in underground forums across the Dark Web.
Data analysis can be used to expose patterns in cyber-criminal communication and to detect illicit behavior in the Dark Web, says Christopher Ahlberg, co-founder and CEO at threat intelligence firm Recorded Future.
Ahlberg in November at Black Hat Europe 2016 in London will discuss how security pros can discover these patterns in forum and hacker behavior using techniques like natural language processing, temporal pattern analysis, and social network analysis.
Most companies conducting threat intelligence employ experts who navigate the Dark Web and untangle threats, he explains. However, it's possible to perform data analysis without requiring workers to analyze individual messages and posts.
Recorded Future has 500-700 servers it uses to collect data from about 800 forums across the Dark Web. Forums are organized by geography, language, and sectors like carding, hacking, and reverse engineering.
'Pattern of Life'
Ahlberg describes the process of chasing bad actors as "pattern of life analysis." This involves tracking an individual, or class of individuals, to paint a picture of their activity and develop a profile on their behavior.
Over the last six months, he has spearheaded research to analyse more than three years of forum posts from surface and deep web. Forums have originated in the US, Russia, Ukraine, China, Iran, and Palestine/Gaza, among other locations.
The research unveiled a series of Cyber-criminal behavioral patterns. These can be used to discover illicit behavior, create points for further branches of research, and figure out how hackers are focusing on different tech and vulnerabilities.
Recorded Future built a methodology for analysts to track user actors' handles as people jump across and within forums, he explains. Discovering patterns starts with attribution, or putting together a profile for one person. The problem is, bad actors often switch between handles to conceal their activity.
"Nobody puts in their real name," he continues. "The issue is, you might track someone and find half of what they're doing is on one handle, and the other half is on a different handle."
He addresses this complication through a process called mathematical clustering. By observing handle activity over time, researchers can determine if two handles belong to the same person without running into many complications.
Temporal patterns exemplify one trend Ahlberg has taken from his observations of hacker activity. "Overall, hacker forums have lower activity on Saturday and Sunday, and peak on Tuesday and Thursday," he says. The times at which criminals are most active can shed some light on their lives and areas of focus. Some forums have a drop in activity around mid-day, a sign that participants could be full-time workers taking a lunch break.
It's also interesting to watch how forum activity relates to industry news. "By looking at forums and how they react to outside events, we can learn more about what they're interested in," Ahlberg says, calling the process "smoking out rats with external events."
For example, a spike in Wednesday activity could be a sign the forum is reacting to patches and vulnerabilities published by Microsoft and Adobe a day prior.
Patch Tuesday, he says, could be driving "Exploit Wednesday."
