nidsbench

Introduction

Nidsbench is a toolkit for testing network intrusion detection systems (NIDS).

A NIDS is a computer security system which detects misuse, attacks against, or compromise of computers connected to a network. They operate by passively examining network packets as they travel over the wire and alerting administrators when they see something unusual or malicious. [1]

Network intrusion detection is still something of a black art - while it is intuitively easy to understand (analogies to burglar alarms, traffic speed traps, etc. abound), the implementation details are often overlooked. For example, in a seminal paper on network intrusion detection published last year, Ptacek and Newsham demonstrated that the vast majority of commercially-available NIDSs are trivially defeated. Fundamental problems in passive monitoring of TCP/IP limit the ability of a NIDS to correctly determine what's actually happening at the endpoint of a traffic stream, and most NIDSs actually do nothing to correct for them. [2]

The goal of the nidsbench project is to provide better tools for evaluating NIDS products and to help standardize a testing methodology for the purpose of objective comparison. Other groups are already working toward the same goal - some industry magazines have their own security test labs (such as InfoWorld and DataComm), IDS shootouts are being featured at industry conferences, and a few research groups have made much headway in the areas of NIDS taxonomy, formal testing environments, reference network attack corpora, etc. [3]

Applications

Nidsbench provides tools to evaluate two measurable NIDS characteristics: performance and correctness. Of course, there are many other features that could be evaluated, but none as objective or easily quantifiable.

Nidsbench includes the following programs to do this:

tcpreplay

Tcpreplay is aimed at testing the performance of a NIDS by replaying real background network traffic in which to hide attacks. Tcpreplay allows you to control the speed at which the traffic is replayed, and can replay arbitrary tcpdump traces. Unlike programmatically-generated artificial traffic which doesn't exercise the application/protocol inspection that a NIDS performs, and doesn't reproduce the real-world anomalies that appear on production networks (asymmetric routes, traffic bursts/lulls, fragmentation, retransmissions, etc.), tcpreplay allows for exact replication of real traffic seen on real networks.

fragrouter

Fragrouter is aimed at testing the correctness of a NIDS, according to the specific TCP/IP attacks listed in the Secure Networks NIDS evasion paper. [2] Other NIDS evasion toolkits which implement these attacks are in circulation among hackers or publically available, and it is assumed that they are currently being used to bypass NIDSs. [4]

idstest

Idstest is aimed at testing the correctness of a NIDS by actually performing the attacks such systems are supposed to detect. In theory, this is no different from what several commercially-available vulnerability scanners do, except that many of them only look for vulnerability symptoms (ex. versions reported in software banners) instead of actually attempting exploits.

Nidsbench does not include a corpus of data to run tests with, nor does it specify a procedure or methodology to use in evaluating NIDSs. We hope that our software is useful to those looking for the tools to instrument such tests, but we have refrained from trying to specify how they should actually be conducted - the rest is up to you!

Distribution

Tcpreplay and fragrouter are available now for downloading.

File	Description	Supported Platforms
fragrouter-1.1.tar.gz	Fragrouter version 1.1	BSD, Linux, others?
tcpreplay-1.0.1.tar.gz	Tcpreplay version 1.0.1	BSD, Linux, Solaris, others?

Contact

Please send all questions, comments, and bug reports to <nidsbench@anzen.com>.

References

1. General NIDS information:
   • Robert Graham's NIDS FAQ
   • CIAC Network Monitoring Tools page
   • SANS/NSA Intrusion Detection Tools Inventory
   • Michael Sobirey's Intrusion Detection Systems page

2. Problems in passive network monitoring:
   • Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection", T. Ptacek and T. Newsham, Secure Networks, January 1998.
   • "Bro - A System for Detecting Network Intruders in Real-Time", V. Paxson, Lawrence Berkeley National Laboratory's Network Research Group, USENIX Security Symposium, 1998.

3. Related NIDS evaluation projects:
   • "Network Intrusion Detection solutions", InfoWorld magazine, May 1998.
   • "Intrusion Detection Systems: Suspicious Finds", DataComm magazine, August 1998.
   • SANS99 ID-Net - a photo of the coordinated attack team members
   • Global Security Analysis Laboratory at IBM Zurich
   • DARPA Intrusion Detection Evaluation program at MIT's Lincoln Labs
   • "Designing IDLE: The Intrusion Data Library Enterprise", U. Lindqvist, First International Workshop on the Recent Advances in Intrusion Detection, September 1998.
   • "Measuring Intrusion Detection Systems", R. Maxion, First International Workshop on the Recent Advances in Intrusion Detection, September 1998.

4. Publically-available NIDS evasion toolkits:
   • Congestant, horizon, Phrack issue 54, article 10, December 1998.
   • BANE, dreck, February 1999.