# 2. The theory behind LogIDS

Intrusion detection, much like any other sectors of the computer security game, is still in its infancy, even if great progress were made in the recent years. But despite these progress, we are still very far from the Hollywood cliché of the network admin being a simple security guard looking at a screen and "seeing" network intrusions that made me laugh so often in various movies. Every seasoned network administrator and security consultant knows that there is no silver bullet for security, and that you can't just stay there and "watch" your network. You have to apply various security tools, apply patches, perform audits, review logs, on top of maintaining the systems. But what if this was all about to change? Or at least a portion of it?

The work around the things presented in this document comes directly from what I've learned and applied in computer security over the last few years and from my previous experiences as a network administrator under different PC-based operating systems (DOS, Windows, OS/2, Novell). LogIDS, LogAgent, ComLog, adsscan and integcheck all come from the same will to increase the overall security of Windows-based networks at an affordable cost in money and human ressources, and provide enhanced intrusion detection capabilities than what was previously the "industry standard". All this work has been inspired from various degrees from different topics such as host bastioning and firewalling, intrusion detection, forensics and incident response, honeypots, along with the work of several individuals that would be too long to name here but who pionneered computer security on the UNIX platform years ago. I have tried to bring to the Windows environment the same philosophy that inspire so many *NIX administrators out there.

So, you might ask, what is the common link between these various topics and LogIDS? Well, first of all, I coud say that they are all related, in one way or another, at increasing computer and network security. Some of these methods are pro-active (establishing a defence), some are reactive (analysing an incident that occurred) and even educational (honeypot technologies). Even if we could discuss about each of these topics in great details separately, I would prefer to focus here on the "common link" I suggested above. Let's look a protection and detection technologies, like firewalls, antivirus, host and network based IDS, they are good at protecting some aspects of your architecture, but cannot handle the whole security needs all by themselves. Also, each of these tools produce logs that must be manually analyzed and interpreted, which is tedious and always after the fact. Besides, all of these logs are all separated from each other, and makes it hard to link things together. For one thing, I hope LogIDS will redefine the term "log correlation". On the other hand, forensics and incident response, do not rely completely on, but are greatly advantaged, by the presence of valuable log data, in condition that it is maintained in a place where we can attest of its credibility.

Honeypots, in a sense, are a variation of the forensics and incident response problematic, in the way that it is heavily focused on the analysis of compromised hosts. Where it gets interesting, though, is that the compromised machines where designed to get broken into, which allows for a level of monitoring that is out of the question in a production environment, for technical and legal purposes. One of the great advantadges of this setup is that it lets the community learn from real-life incidents, monitoring the activities of unsuspecting black-haters. Even in the highly monitored environment that is a honeypot, it became clear that certain circumstances required some new strategies or some new tools to develop, profiting to the security community as a whole, while learning the mechanincs of different real-life attacks. The analysis of a compromised honeypot also involves, but is not limited to, the analysis of various log files, which are generally richer and more diversified than what is met in an average production environment.

LogIDS, along with the other tools mentionned above, are built from the learning of each of these fields. The strategy behind it is to implement various level of security controls, some of which are probably already present in your network, have these security measures deployed on every host on the network, centralise the logs produced by these various tools , and analyse as soon as possible the log data as it arrives on the central host. Some tools provided by SecurIT Informatique Inc. fit in the log evidence producing category of tools (ComLog keeps a log of cmd.exe sessions, adsscan scans for ADS, a little known but dangerous feature in terms of system security, and integcheck that produces integrity checking of your file system in a way similar to Tripwire). LogAgent 4.0 Pro also produces some data, related to aspects of the system that are analyzed in order to determine a succesful intrusion, like rogue running services, open shares that should not be, items placed in the startup configuration (common for trojans). All versions of LogAgent also specializes in monitoring and forwarding of ASCII log files, like the one produced by a great number of security tools around, along with the events from the Event Viewer, all of this in real-time. This allow for the easy integration of other security solutions not covered by the tools I provide (and there are many!) that could enhance your network security, such as network-based IDS like Snort, a personnal firewall, an antivirus, and just about any other kind of security tool I cannot think of here.

This gives us a scenario where we have a good deal of log information to treat in order for all this to make sense. Some of this data are mere warnings of attempted illicit activities that were easily dealt with by the reporting software (like a cleaned virus, a blocked port scan), while some of this data was traditionnaly looked only after a confirmed incident occured, was produced during intrusions but could not stop it (like Snort, for example), or was simply unavailable (in the case of ComLog, which is very precious in the context of forensics analysis). The combination of using LogAgent with a variety of tools provide you with the ability to have all of this data available as it is procuced, which is much more better than, let's say, a weekly analysis. The problem is to present an interface where to make sense of all this heterogeneous data, and effectively detect intrusions in the case that one part of your security architecture drops the ball.

LogIDS is my attempt at solving this problematic. I tried to move away from the traditional interface, where we are simply presented with a list of log entries, that we have to decipher as they fly. LogAgent is a simple version of this crude log display interface, but let's be frank, most of all the log analysis software (real-time or not) are more or less a variation of the same interface of log lones after log lines. Some of them will let you click on a log to enter the details and link it to a database, but in my opinion, even if I think these features are mildly cool, I think they are cumbersome to navigate between the screens when time is critical and you have to wait after the web server's response (who may not be able to reach you because you're flooded, for example). And most of all, most of these tools are single-application oriented only, meaning they are designed to monitor either the Event Viewer, either Snort, either ZoneAlarm, but nothing offers a really united view of what is going on.

So I designed an interface that lets you represent at least a logical representation of your network environment, where you have not 1, but as many as you have defined items text windows to display log data. I hope this representation will help to better understand the interactions between the various systems when they are involved in a complex attack. LogIDS also provides with an easy to configure filter module to allow for pretty much all kind of ASCII log files, and a rules module to modify LogIDS behavior at your will. It is designed specifically so you can adapt it to your environment, giving you all the flexibity to automate some of the analysis aspects with the rules file. Further most, as we will see later, even if an intrusion occurs, LogIDS will still detect it and unless you're either blind, deaf or dead, there should be no doubt in your spirit about if you've been hacked or not when it happens. In the case you do get hacked into, the difference now is that you know about it as soon as it occurs, and lets you take rapid action to solve the issue. We will see in the following chapters how to configure LogIDS and how it can help achieve these goals. Also, I thought it was important to have an Open Source version of such a tool, because it would not help LogIDS nor the security community to keep it under shadows: it is clear to me that this tool has potential to grow far more than I alone can conceive. Still, I hope this tool will also be recognized as a key piece in a security architecture in the commercial market, which will sustain in the long term the future development of LogIDS.
1. Introduction
3. Centralising your logs