The Role of Metadata Analysis in Information Security

The Role of Metadata Analysis in Information Security

POSTED BY Tom Cross on 08.05.2014

Last year Edward Snowden revealed that the NSA is collecting the telephony metadata of millions of Americans. This revelation has sparked a debate about the power of metadata. Supporters of the program have attempted to reassure the public that their privacy is not being violated because the content of their communications are not being collected. On the other hand, opponents of the program have highlighted the deep insight that can be gained from metadata analysis.

For example, shortly after news of the program became public, Senators Dianne Feinstein (D-Calif.) and Saxby Chambliss (R-Ga.) held a press conference to address the issue.

Senator Feistein said that "As you know, this is just metadata. There is no content involved. In other words, no content of a communication." Senator Chambliss later added that "All these numbers are basically ferreted out by computer, but if there’s a number that matches a terrorist number that has been dialed by a U.S. number or dialed from a terrorist to a U.S. number, then that may be flagged. And they may or may not seek a court order to go further on that particular instance."

Edward Felten is a professor of public affairs and computer science at Princeton University, and Chief Technologist at the Federal Trade Commission. In December of 2013, the ACLU filed a lawsuit challenging the legality of the NSA telephony metadata program, and Professor Felton filed a declaration in that lawsuit describing the power of metadata.

Professor Felten points out in his declaration that telephony metadata is relatively easy to aggregate and analyze, as opposed to content. He writes that "Telephony metadata is, by its nature, structured data. Telephone numbers are standardized, and are expressed in a predictable format… The structured nature of metadata makes it very easy to analyze massive datasets using sophisticated data-mining and link-analysis programs… The contents of calls are far more difficult to analyze in an automated fashion due to their unstructured nature… It is not surprising, then, that intelligence and law enforcement agencies often turn first to metadata… Telephony metadata can be extremely revealing, both at the level of individual calls and, especially, in the aggregate."

At Lancope, this discussion is interesting to us because we purpose-built the StealthWatch System to collect and analyze metadata - not Telephony metadata, but Internet metadata. Our customers buy the StealthWatch System in order to monitor their own networks, often for the purpose of protecting their own privacy. Private computer networks in the United States and around the world are being targeted by sophisticated, state sponsored network intruders who are interested in infecting them and spying on the people who use them. What role does metadata collection and analysis have to play in detecting these attacks?

Sometimes, we hear the objection that it would be better to collect and monitor content, instead of focusing on metadata. The fact is that metadata is more cost effective to aggregate and analyze than content, as Professor Felten made clear in his declaration. Most organizations cannot afford to collect and store more than a few days of full packet capture. Packet capture collection also requires the installation of dedicated equipment at each collection point. This isn't true for Internet metadata. Every piece of equipment in a typical computer network – every router, every switch, and every access point usually has the built in capability to record and forward metadata to a collection point. Internet metadata collectors can observe the entire network from wherever they are located – they do not have to be installed at every location that needs to be monitored, and they can economically store the records of many months of network activity. This is why our customers often tell us that they have better visibility into whats happening on their networks with the Stealthwatch System than with any other information security monitoring tool that they have.

That visibility can have a huge impact. Senator Chambliss referred to tracking calls to phone numbers belonging to terrorists. Our customers often track Internet connections to IP addresses where malware command and control systems are hosted. Once a connection to a known command and control location is established, metadata analysis can reveal all of the attacker's tracks within a network as they have accessed different systems and infected them. Whats more, sophisticated analytics built into the StealthWatch System can identify hosts within a network that are behaving strangely – hosts that fit the profile of a system that may have been compromised.

At Lancope, we believe that metadata is a powerful tool that can be brought to bear on the problem of protecting computer networks against advanced attackers, and we think that every organization with a computer network should consider metadata collection and analysis to be a key component of their overall computer security strategy. This is all a part of what the industry is calling context-aware security – an approach to protecting networks that involves the use of supplemental information to improve security decision making. As incident response becomes a continuous business process, that contextual information is the key to ensuring that the right security decisions are being made.

In order to promote discussion about this topic, Lancope is distributing t-shirts at Blackhat USA 2014 that are printed with a QR-code leading to a website that we've established that contains links to some information resources. We're also suggesting that people use the Twitter hashtag #justmetadata to post relevant thoughts and content.

If you are attending Blackhat USA 2014, please come by our booth (#1115) and get a t-shirt, and don't miss my talk on Wednesday morning titled "The Library of Sparta." Greg Conti, David Raymond, and I will be discussing the ways that military doctrine can help provide us with new perspectives for thinking about the problem of computer security and fighting advanced adversaries.