Integrating StealthWatch System with SIEMs
In today’s world of Advanced Persistent Threats (APTs) and Insider Threats the need for visibility into network traffic has never been greater. Although originally used for network management, NetFlow seems to be a purpose built technology for the Security Analyst, providing a scalable way to monitoring all host-to-host communication and to answer those critical security questions such as:
- Who is talking to whom?
- What applications are being used?
- When did the communication occur?
- Where did the communication take place?
- Did it go through a firewall and if yes, then why was it permitted?
- How long? How much? How frequently?
The rich data set contained with a NetFlow record can also be used to establish behavior baselines and alert on anomalistic and/or suspicious activity in the network. Due to the richness of this data set the question often arises on how to best integrate NetFlow into an existing Security Information and Event Management (SIEM) deployment.
NetFlow Security Monitoring at Scale
Before going too far down the path of just sending NetFlow to an existing SIEM deployment it is important to understand the NetFlow protocol, the characteristics of your network and what your goals are.
The NetFlow Protocol
I’m not going to go too deep into the NetFlow protocol here, as that can be an overly large topic for a single blog post; but, there are a couple of items about the nature of the protocol and the exporting devices to take into consideration when designing a monitoring solution:
- NetFlow is unidirectional – meaning that for a single client to server transaction there will be a NetFlow record generated to represent the client to server communication and a NetFlow record generated to represent the server to client communication
- NetFlow records are generated on timers – meaning that a NetFlow record is generated by the exporting device on a timeout event. Timeout events fall into two categories active (packets are being forwarded through the device) and inactive (no packets). Best practice recommendations are to set the active timer to 60 seconds and the inactive to 15 seconds.
- NetFlow records are generated by every NetFlow enabled device in the packet path. In this design scenario I am assuming that the goal of the NetFlow Security Monitoring program is to provide complete visibility into all host-to-host communication which will mean enabling NetFlow everywhere that is physically possible.
The reason that I pointed these design considerations out is to illustrate the fact that NetFlow can add up quickly – in fact each of the above considerations multiply the size of the deployment quickly up into the tens to hundreds of thousands of NetFlow records per second, greatly increasing the data collection rate and storage requirements of a NetFlow collection system.
Through the processes of stitching and deduplication, Lancope’s StealthWatch System minimizes the effect that the above considerations have on the collection system. Stitching refers to uniting the unidirectional NetFlow records (client->server and server->client) into a single record. Deduplication refers to the process of identifying NetFlow records that logically represent the same flow but come from different devices without losing relevant information such as interface statistics. The result being that a single line item – representing the stitched and deduplicated flow - is stored in the database (the Flow Table), greatly reducing the required disk size from storing the raw NetFlow data as well as creating a system that is more scalable and easier to use and query the dataset.
Furthermore, StealthWatch has integration capabilities with identity aware devices, such as the Cisco Identity Services Engine (ISE), the StealthWatch IDentity appliance, firewalls and other network devices, to attribute the IP Address continued within the NetFlow record to a username and a device - this information is also stored in the Flow Table.
Integrating NetFlow into the SIEM
Most SIEM’s, while publishing support for the NetFlow protocol, do not support the operations of stitching and deduplication. This means that enabling NetFlow at scale in an enterprise network can quickly overwhelm a SIEM as a NetFlow record is considered an Event in SIEM terminology – consuming valuable processing, license and disk space and making the collection of NetFlow at scale in a standard SIEM a costly endeavor. It is much more efficient and cost effective to deploy a dedicated NetFlow collection system, like the StealthWatch System, and integrate it into the SIEM deployment. The figure below illustrates at a high-level StealthWatch-SIEM integration.
In the above image the StealthWatch Management Console (SMC) is sending security events such as the behavior alarms (e.g.. High Concern Host) or policy (e.g. Host Lock) violations using one of the many response actions configured through the Response Management configuration dialog (see below).
In addition to having StealthWatch send security events to the SIEM it is also possible for the SIEM to call APIs available on the SMC. Effectively these APIs can expose the entire database of NetFlow data allowing a SIEM (or other third party tool) to have controlled access to the NetFlow data without suffering the drawbacks of having to collect it at scale. For example integration can be seen below making use of the getFlows API on the SMC from inside the Splunk console.
If an enterprise is serious about leveraging the power of NetFlow for security purposes then it is necessary to use a purpose built NetFlow collection system – the nature of NetFlow protocol can quickly lead to massive amounts of data that require special handling in order to leverage the benefits of the data for security purposes.
Lancope StealthWatch System is purpose built for the handling of large volumes of NetFlow to efficiently analyze the data to identify anomalistic activity. The StealthWatch System can then be integrated with other Security Monitoring Technologies to increase the value of NetFlow, StealthWatch and the other monitoring technologies to create a scalable NetFlow Security Monitoring solution.
Learn more about how StealthWatch Systems works with SIEMs here.