SécurIT Intrusion Detection Toolkit (SIDTk) 1.0 
By Floydman, floydian_99@yahoo.com
October 19th, 2003

This paper is available online at http://securit.iquebec.com

This paper can be freely distributed and reproduced, as long as correct credentials are maintained, and that no modifications are made to this file.  For corrections, suggestions or comments, please send me an e-mail.

Abstract

The goal of this paper is to present the SécurIT Intrusion Detection Toolkit (SIDTk) 1.0, a set of tools made in Perl for improving the intrusion detection capabilities of current Windows network architecture.  This is achieved by providing an expandable set of independent modules specialized at inspecting and analyzing specific parts of the system for obvious hints of intrusion.  The idea behind these modules comes from combined techniques gathered over the years by experts in forensics and honeypots.  As such, some of these modules are standard file scanners, while others can be seen as automated forensics modules.

Preface

The whole LogAgent/LogIDS internal architecture have been reworked since LogIDS initial release, in order to obtain better performance and greater flexibility, including the ability to add additional logging/intrusion detection modules.  This has lead to the release of the SIDTk (SecurIT Intrusion Detection Toolkit), which is presented here.  LogAgent 5.0 Open Source code is equivalent to LogAgent 4.0 Pro, minus some of the logging code that was introduced in LogAgent 4.0 Pro.  LogIDS 2.0 Open Source hasn't suffered much changes in this release, and yet the Open Source contribution of the whole software release gained much more code from the previous Pro versions.  It is in the form of the SIDTk 1.0 that this extra code, removed from LogAgent4.0/LogIDS 1.0 Pro, has made its place in the Open Source world.  Note that this software package is completely Open Source, as opposed to the rest of our software which have both types.  Only the ability of running it as a service under LogAgent is restricted by LogAgent's license.

Targeted audience

This document is presented to anyone who has interests in computer security, NT/2K Administration, computer and network monitoring, intrusion detection, forensics, honeypots, Perl programming and computing in general.

Table of contents

1. What is the SIDTk 1.0?
2. ADSScan 1.0
3. IntegCheck 1.1
4. LogServices 1.0
5. LogShares 1.0
6. LogStartup 1.0
7. LogUser 1.0
8. LogProc 1.0
9. Generic use
10. Conclusion
Appendix A. Code shell

1. What is the SIDTk 1.0?

The SécurIT Intrusion Detection Toolkit, hereby referred to as SIDTk, currently at its release 1.0, is a collection of tools aimed at intrusion detection, as its name indicates.  This comes from good ideas that were badly implemented in the LogAgent 4.0/LogIDS 1.0 Pro release.  With this release, LogAgent 4.0 Pro had the ability to produce some logs of its own, related to some parts of the system's configuration that are usually places where evidence of an intrusion can be found, like the running services, open shares and the startup configuration (where trojans can place themselves to run automatically at each boot-up).  This was also accompanied by IntegCheck/Hashgen 1.0 and ADSScan 1.0, that are a file integrity checker (like Tripwire) and an alternate data stream scanner, respectively.

On the other side, LogIDS 1.0 Pro had analysing capabilities that allowed it to interpret this data and to find invalid entries from a configuration file.  While the concept actually worked, the implementation was bad for several reasons.  This is now fixed with the SIDTk 1.0 release.

One of the reason why the implementation was badly designed is that the extra analysis code was run only at LogAgent's startup, which is not of great help at detecting anything on a machine is almost always on, like a server for example.  Another problem was that this made way too much logs to analyze for no reason at all by the LogIDS console.  Most of these logs worked by matching the data in it against a file containing what were considered as valid items.  If the item reported in the log was not in the file, then it was brought as an alert.  But in fact, the console should only have to analyze and report on invalid items, the pre-analysis stage should be done on the host itself, and then only invalid items are reported to the LogIDS console, reducing greatly the workload and thus improving the performance of it.  This is what happens with the SIDTk.  Besides IntegCheck and ADSScan, the rest of the modules are actually a combination of the data-gathering code previously present in LogAgent and the analysis code previously present in LogIDS, that perform both tasks at once, and forwarding to the console only items out of specification.

There are actually some new modules that have been added to the SIDTk 1.0 that were not present in the LogAgent 4.0/LogIDS 1.0 Pro code, which leads to the last problem this release aims to fix.  To include new modules, I was faced with having to release a new binary of LogAgent every time I wanted to include some new code.  This was not really efficient, and with this new design architecture, it is now easy to add new modules as they are completely independent of LogAgent and LogIDS code.  The SIDTk only shares the use of LogAgent's config.txt files, which contains information about the format and destination of the logs.  New modules can be created at will, and a shell structure is provided in appendix so you can write your own modules that work on the same specifications as the rest of the SIDTk.

By taking the code out of LogAgent/LogIDS code base, I actually fixed all of these problems at once.  The modules can be run as normal command prompt programs, or be added in the file modules.txt, that is used by LogAgent 5.0 Pro when running as a service to include these modules.
New modules can be released and included in the toolkit without having to redeploy the rest of the tools.  The amount of logs analysed by the console is much reduced, and is focused only on invalid items.  And finally, this code have been taken out of the proprietary bookshelf and been provided as Open Source for the rest of the community.

Now, let's review one by one the tools making the SIDTk 1.0.

2. ADSScan 1.0

Originally released with LogAgent 4.0 as an Open Source add-on tool, it now makes an integral part of the SIDTk.  This program is a file scanner that will spot alternate data streams, which can be shortly described as a hidden file within a file, and which cannot be seen by normal ways by the system's user.  This is present on Windows NT-based systems, and was previously undocumented.  As an example, let's say we have a file called test.txt, which is a very normal ASCII text file.  Now, if I wanted to hide the binary file root.exe into an alternate data stream, I would end up calling my program test.txt:root.exe.  In Windows explorer, or from 'dir' commands from a DOS box, all you would see is 'test.txt'.  Now, what is funny is that if you run the ADS test.txt:root.exe, you will see 'test.txt' as the running process in the tasks list.

This program is a standalone program, and is called by its binary name adsscan.exe.  It takes the file config.txt from LogAgent (only config.txt is necessary, not LogAgent itself, more later about this file).  ADSScan will scan every file on every NTFS partition present on your hard drive(s) and will send an alert when one is found.  The scanning process has actually been slowed down from the version shipped with LogAgent 4.0, even if there is no version change for ADSScan.  This was done so that ADSScan, along with IntegCheck, doesn't eat up all the system's resources when run.  It will take longer to run, your hard drive will seem to work a lot, but in fact your system is under less stress in this version that the previous.  There are however some phases where the software could not be slowed down, and you may experience overall system performance similar to what is encountered when running an antivirus scan on a machine.  These phases are only temporary though, and will not last the whole process.

Here is what an ADSScan log item looks like (adsscan.log):

E:/WINNT/lanmannt.bmp:test.txt,(77 bytes)

The log itself is made of two entries, the alternate data stream filename and its size, separated by a comma.  If you plan on using LogIDS 2.0 (any version) with ADSScan, besides the appropriate fields definition for the items in config.txt, define two fields that can be named 'adsfile' and 'adssize', for example.

3. IntegCheck 1.1

Like ADSScan, IntegCheck was also shipped with LogAgent 4.0, and was also slowed down with this release, for the same reasons (except that IntegCheck process is more linear than ADSScan in terms of performance), but the reason why it has been upgraded to version 1.1 is different.  The previous version was actually a two-part tool, HashGen and IntegCheck.  IntegCheck is a file system integrity checker, or a host-based intrusion detection system, like many others found on the net these days.  HashGen was the part to run for producing a hash table of valid binaries that the system is matched against when IntegCheck is run.  With IntegCheck 1.1, both binaries have been merged into a single file, which makes for easier handling of the program by the user.

The program is a standalone program, which also uses the file config.txt (see later).  You configure it with the file hash_conf.txt, which is made of entries designating the files and directories you want to submit under IntegCheck's watch.  The program also uses the file hash_log.txt, which is created after IntegCheck has run at least once.  This files contains the list of files under IntegCheck's watch, along with a MD5 and SHA1 hashes of these files, which is used to identify compromised or corrupted binaries of configuration files.

The program can be run either by calling the executable's name integcheck.exe, which will run the program in default mode, or by typing "integcheck.exe hashgen" to force the generation of a new hash_log.txt file.  In default mode, IntegCheck will generate hash_log.txt if it is not present, or will process to scan the system if the file is present.

Here is what can look like hash_conf.txt:

c:\io.sys
c:\config.sys
c:\autoexec.bat
D:\winnt\
D:\winnt\system32


And here is what can look like hash_log.txt:

c:\io.sys,WhgWcEK9+YI+9/cMp6+AWw,IpJPk90PnqakYkzN0bvN9etDowg
c:\config.sys,BpOwXm44AT8o9Ow2s9YgyQ,PK4IcY9DenetZohnyTg3f2HjNro
c:\autoexec.bat,UBm8EIkIjX24JoD8oFwZrA,U/b3LE2BizYo6tPPKaIAFugEy+Q
D:\winnt\Active Setup Log.txt,Hn1NH5Q6otJA4EBusZt3Pg,dVwHjnbtCH5A6RmZrDp6MCHzrqs
D:\winnt\black16.scr,Th31EVfM17xkopZoX3YGjg,nSrJyduPgMamxOvsCvMpyOKPzqY
D:\winnt\clock.avi,u1FpR3aPuwW0GiSH8gBxbg,IXOs9RevijcKdEn4MS/rBaBuuxc
D:\winnt\clspack.exe,ybyhGHw86yfDEezqAEzxzA,PFetax8lHEP3FSCYEAhckwAFb1w
D:\winnt\config.txt,XBwVtPn5V1zmjG3LqN/E4Q,PGfMWiVysFHdldSAcMpoaHozFEo
D:\winnt\control.ini,1B2M2Y8AsgTpgAmY7PhCfg,2jmj7l5rSw0yVb/vlWAYkK/YBwk
D:\winnt\drwtsn32.log,onWklSOc+k8Jgi4ijSzpAg,uZpbenqeNYE+Gh9ZS6A0zrnusl8
D:\winnt\EXPLORER.EXE,SDTQ9cRYayLW07ruQBgd3w,32eLW1q4DK0XKM9cAsjKNQZcwos
(snip)
D:\winnt\system32\$winnt$.inf,XfQ7h6NlFGtVulzgQ0cRAQ,iwr1Nho1QxEG0ECJ7l01Bhx5ELw
D:\winnt\system32\12520437.cpx,Cg/rnrKL3ozYNXFjQ7A7FA,oEDUQO1xrY9pn/a5K+C1XE1W3LY
D:\winnt\system32\12520850.cpx,1prgV82C0E7n0xGAmr77Kg,BlA5reG87mulTA2cZSegM0MJjJQ
D:\winnt\system32\abmsg.dll,JAxGt6S9BzqWoY18iEmNaA,sFRaL2BPnC+H+q6M8wur/XDQQBU
D:\winnt\system32\access.cpl,3VWCYcT7eZnhrc+HuzTyrA,Ux4HQ+vLOPYSlyrOaQsC6Zyy9sQ
D:\winnt\system32\ACLEDIT.DLL,+G4bnkFYBysY0+RKQJP20g,C3raYaI2Ner60SHBdmHMLRhFryE
D:\winnt\system32\activeds.dll,AHK0G3yYYUwWY3Bw9xREWA,ATM/6PZ5y/Aec5Ai/TlWYQqSBA0
D:\winnt\system32\activeds.tlb,2Pf+jFewFnlOCPrtBf9AOQ,93ICBCMJEOidmCNjs3/rT7kZERg
D:\winnt\system32\actmovie.exe,OzjQgWG7kKyOdd6MBDaRmg,qfHDjWpnvJIBSBPZFlBoAG7tXMk
D:\winnt\system32\ACTXPRXY.DLL,vpRmNCVy8vwn8XdO6wPiEw,5nmZwDx3sRBkDQlnbmGeUcF6t+8
D:\winnt\system32\addgrpw.exe,MZ7E0vSC66/5FRy3VXE/UA,eU3apcyPWMXpjX+LIqv8BJsZp5c
D:\winnt\system32\addusrw.exe,PlmacujBWL2DZQFBQkxJYw,UFDJJgyCU1FjlrLmdpk2ygo3I3I
D:\winnt\system32\adme.dll,gSPUq8EwxbyQtk0Y4m8tlw,KCDMwWAVzQ+K+zlaE5YnnWb5sDI
D:\winnt\system32\admwprox.dll,0YzXy9FJXH+r9RoBfgy+NA,5toYFfHwe3cOXMbBIEB5HGsXzFM
(snip)

Note that IntegCheck will not recurse into sub-directories.

Here is what Integcheck log entries (integrity.log) would look like (stripped of the extra fields defined in config.txt):

Integrity check failed; new file D:\winnt\a.txt added to system.
Integrity check failed for missing file D:\winnt\Active Setup Log.BAK; system file deleted.
Integrity check failed; new file D:\winnt\AdminMisc.dll added to system.
Integrity check failed; new file D:\winnt\API.dll added to system.
Integrity check failed; new file D:\winnt\Daemon.dll added to system.
Integrity check failed for file D:\winnt\system32\mfc42u.dll.
Integrity check failed for file D:\winnt\system32\Msvcrt.dll.
Integrity check failed for file D:\winnt\system32\__MMtmp_.


As you can see, it is made of a single field containing the whole message (the logs are comma-delimited).  At least, this is how you should treat it if you plan on using LogIDS with IntegCheck.

4. LogServices 1.0

LogServices (logservices.exe) is a new module stripped out of LogAgent 4.0/LogIDS 1.0 Pro, and is designed at spotting rogue running services.  This module is pretty straightforward, and is designed along the same lines as the next modules.  Besides config.txt, this module takes its configuration from the file services.txt, which simply contains a list of the services allowed to run on the machine.  If a running service is found and is not present in the text file, an alert is produced.

Here is what a services.txt file may look like:

Alerter
EventLog
Server
Workstation
LogAgent 5.0 Pro Service
Messenger
TrueVector Basic Logging Client
Plug and Play
Protected Storage
Remote Access Autodial Manager
Remote Access Connection Manager
Remote Procedure Call (RPC) Service
Task Scheduler
Telephony Service
TrueVector Internet Monitor
Computer Browser
Content Index
IIS Admin Service
License Logging Service
TCP/IP NetBIOS Helper
MSDTC
FTP Publishing Service
MySql
Microsoft NNTP Service


As you can see, it is only made of the Label name for your services as you would see them from the Service manager.  You can use the output of services.log to build your config file.

And here is what an alert may look like in the file services.log:

Unknown service detected,NT LM Security Support Provider,NtLmSsp,Running,LocalSystem,D:\WINNT\System32\SERVICES.EXE,Manual
Unknown service detected,Microsoft SMTP Service,SMTPSVC,Running,LocalSystem,D:\WINNT\System32\inetsrv\inetinfo.exe,Automatic
Unknown service detected,Spooler,Spooler,Running,LocalSystem,D:\WINNT\system32\spoolss.exe,Automatic
Unknown service detected,World Wide Web Publishing Service,W3SVC,Running,LocalSystem,D:\WINNT\System32\inetsrv\inetinfo.exe,Automatic

As you can see, it is a 7-field comma-delimited line, comprising fields for the message, the service label (the one you should base your services.txt file on), the internal service name, its current state, the account it is setup to run under, the binary full path and the startup setup (manual or automatic).  If you plan on using LogIDS with LogServices, you could use the field names message, label, name, status, account, binary and startup respectively, for example.

5.0 LogShares 1.0

Like LogServices, LogShares was present in LogAgent 4.0 Pro, and has been transferred in the SIDTk much in the same way.  The principle is exactly the same, with the exception that it is applied on Open Shares instead of the Running Services.  Besides the common use of config.txt, LogShares takes its configuration from the file shares.txt and will send alerts in the file shares.log.

Here is what a shares.txt file may look like:

Log
Log$
ADMIN$
C$
D$
G$
H$
I$
IPC$
D
E$
F$

And here is what a plain shares.log file may look like:

Non-allowed share detected,print$,Everyone,R-X----,D:\WINNT\system32\spool\drivers,Printer Drivers
Non-allowed share detected,print$,Administrators,R-XD--A,D:\WINNT\system32\spool\drivers,Printer Drivers
Non-allowed share detected,print$,Power Users,R-XD--A,D:\WINNT\system32\spool\drivers,Printer Drivers
Non-allowed share detected,TESTLOG$,Everyone,------A,D:\LogIDS2\Log,
Non-allowed share detected,Z11,CREATOR OWNER,R------,Lexmark Z11 Series ColorFine,LocalsplOnly,Lexmark Z11 Series ColorFine
Warning:  Autorun.inf file detected on drive root testlog!!,\\DARKSIDE\testlog\autorun.inf,Everyone,R-XD--A,D:\Dev\Logids2\LogIDS\Log,
Warning:  Autorun.inf file detected on drive root TESTLOG$!!,\\DARKSIDE\TESTLOG$\autorun.inf,Everyone,R-XD--A,D:\Dev\Logids2\LogIDS\Log,

AS you can see, this file is a 6-fields comma-delimited text file, with the field definition going like this: generic message, share name, group access, permission list, share path, and comments (if any).  Again, if you wish to use it with LogIDS, you could use field names like message, sharename, group, perms, sharepath and comments respectively.

6. LogStartup 1.0

LogStartup is the last module that is derived from LogAgent 4.0/LogIDS 1.0 Pro.  This one works like the previous two, scanning the startup configuration of the system, and comparing it against a file of valid entries.  This one is a little bit more complicated however, since the "startup config" is actually taken from more than one place (the startup folder and some places in the registry), and the syntax may be a bit tricky, so it can be awkward from your part to guess it right the first time.  The easiest may be to run the tool with an empty config file on one of your base system with a config like you see on your network to see what pops-up, and use this data to build your startup.txt file.

Here is how a startup.txt file may look like:

userinit,nddeagnt.exe
SystemTray:SysTray.Exe
AVG_CC:D:\Program Files\Security\Grisoft\AVG6\avgcc32.exe /startup
WinampAgent:"D:\Program Files\Multi Medias\Winamp\Winampa.exe"
BrowserWebCheck:loadwc.exe
SchedulingAgent:mstinit.exe /logon
Cookie Crusher:d:\program files\security\Cookie Crusher\ccwatch.exe
internat.exe:internat.exe
GetRight Monitor.lnk:D:\Program Files\Utils\GetRight\getright.exe
Microsoft Office Shortcut Bar.lnk:D:\Program Files\Microsoft Office\Office\MSOFFICE.EXE
PGPtray.lnk:D:\Program Files\Security\PGP\PGP50\PGPtray.exe
ZoneAlarm.lnk:D:\Program Files\Zone Labs\ZoneAlarm\zonealarm.exe

And here is what may look like the output file startup.log:

ALERT! Unknown item inserted in the startup configuration,[Checking LMachine/SOFTWARE/Microsoft/Windows/CurrentVersion keys],  - Run=PrinTray:D:\WINNT\System32\spool\DRIVERS\W32X86\2\printray.exe
ALERT! Unknown item inserted in the startup configuration,[D:\WINNT\profiles\Administrator\start menu\programs\startup],Administrator=WinMySQLadmin.lnk:C:\mysql\bin\winmysqladmin.exe

This is a 3-field comma-delimited test file made of the following items: a generic message, location of invalid item (registry hive or directory), followed by an item comprising ' - ' and a sub-hive name for registry entries, or a sub-directory name for file entries, followed by an '=' sign, and the full path of the executable (or the full path a shortcut is pointing to) identified as invalid.  If you want to use this file to build your startup.txt file, simply drop everything left of the '=' sign, '=' sign included.  This way, you will be sure to have the correct syntax for what the program is expecting.  If you wish to use LogStartup with LogIDS, you can name your fields message, location and item respectively.

7. LogUser 1.0

LogUser 1.0 is a new module fresh with the SIDTk 1.0, although it is based on the same structure as the last three we just reviewed.  Along with the common config.txt file, this file takes its config from the file user.txt and produces its output in the file user.log.  This module is targeted at detecting rogue usernames on a system's SAM database (the user database).  This module works for local users only, so to monitor Domain level user accounts, it simply needs to be run on the PDC/BDC of your domain.

A sample user.txt file may look like this:

toto
tata
Administrator
Guest
IUSR_TESTBED
IWAM_TESTBED
LogAgent_USR

The output produced in user.log may look like this:

Unknown user detected,test,NONE,
Unknown user detected,dummy,NONE,dummy

It is a 4-field comma-delimited text file made of the following fields: generic message, username, login script and the user's full name (if any).  If you want to use LogUser with LogIDS, you could name these fields message, username, script and fullname respectively, for example.

8. LogProc 1.0

Last but not least, LogProc 1.0 is also based on the same structure of the last tools we just covered.  Although, this one can be tricky to configure also.  This module is targeted at detecting rogue processes running on the system, like trojan horses or other hacking tools.  Like the other modules based on the same model, it will take its configuration from a text file, called processes.txt, and will send its output in the file processes.log.  The difficulty here is in getting all the right processes into the config file to avoid false negative.  One of the best way to achieve it is to run all the programs you expect to be running on your systems and run LogProc with an empty config file against it, but the problem here is in thinking about all the programs that are subject to be used in your environment.  But I think it can be achieved.  For example, when I made my testing, I configured it so that it would include everything I was expecting to run at test time.  But it happened that I forgot to close the calculator tool, which produced an alert for calc.exe.  Now, I'm sure the same could happen to various Windows tools like notepad.exe, clock.exe, mspaint.exe, etc, but also with some Windows Management tools, like usrmgr.exe (user manager) or taskmgr.exe (task manager).  So the problem here is correctly identifying and including every valid binary you could normally expect, while determining if some binaries are better left aside in some circumstances (for example, you would not expect a normal user to run the user manager on his machine, but maybe a support person is there fixing something, so do we include this file in the list or not?).

Here is a sample of what processes.txt may look like:

System Idle Process
smss.exe
csrss.exe
winlogon.exe
services.exe
lsass.exe
RpcSs.exe
pstores.exe
vsmon.exe
minilog.exe
nddeagnt.exe
Explorer.exe
SysTray.Exe
logserv.exe
logschtasks.exe
adsscan.exe
integcheck.exe
logservices.exe
logshares.exe
logstartup.exe
loguser.exe
logproc.exe
log50pro.exe
logids20pro.exe
avgcc32.exe
Winampa.exe
loadwc.exe
internat.exe
getright.exe
PGPtray.exe
zonealarm.exe
EditPad.exe
iexplore.exe
msipcsv.exe
MDM.EXE
cm_.exe
cmd.exe
MSTask.exe
Perl.exe
spoolss.exe
msdtc.exe
cisvc.exe
llssrv.exe
mysqld-nt.exe
inetinfo.exe
cidaemon.exe
ZONEAL~1.EXE
calc.exe
taskmgr.exe
rundll32.exe
snort.exe
NOTEPAD.EXE


And here is what processes.log may look like:

Unknown process detected,USRMGR.EXE,D:\WINNT\system32\USRMGR.EXE,TESTBED\toto
Unknown process detected,mspaint.exe,D:\WINNT\system32\mspaint.exe,TESTBED\toto

As you can see, it is a 4-fields comma-delimited text file, made of the following fields: a generic message, the binary filename (this is the field used in processes.txt), the binary full path and the username it is being run under.  If you plan on using LogIDS with LogUser, you could name these fields message, binary, fullpath and username respectively, for example.

9. Generic use

All of these tools have been designed to be run as normal command-line programs, which means you can launch them from a command prompt, a batch file or a scheduled job, for example.  All of these programs have to be in the same directory as their respective configuration files, along with config.txt that comes from LogAgent.  If you don't use LogAgent, it is not a problem, you simply need the file config.txt.  All these tools are called by their executable's name, excepted with IntegCheck which can also take 'hashgen' as a parameter.

The file config.txt is made like this:

LOGIP= Put Y if you want to append the IP address of the local machine to your log
LOGHOST= Put Y if you want to append the hostname of the local machine to your log
LOGUSER= Put Y if you want to append the username to your log (not useful when run as a service)
LOGDATE= Put Y if you want to append the date of logging to your log
LOGTIME= Put Y if you want to append the time of logging to your log
SHOWCONSOLE= Put Y if you want to display the logs on the console, put N when run as a service for better performance
LINEPRINT= Put Y to send the output to LPT1
LINEPRINT BUFFER= Put the number of lines you want to buffer before it is sent to the printer (a sorry emulation for line printing), this line is not taken into account when LINEPRINT is set to N

Then followed by a list of destination directory where you want to send your logs:
C:\dir1\
\\server1\share$\
...

or alternatively, simply type
NULL
as the destination if you only want the output to be printed on the screen.

Here is a sample config.txt file:

LOGIP=Y
LOGHOST=Y
LOGUSER=Y
LOGDATE=Y
LOGTIME=Y
SHOWCONSOLE=N
LINEPRINT=N
LINEPRINT BUFFER=40
\\servername\shareddir$\

Now, unlike LogAgent, these programs are not constant monitoring programs, they are a one-shot deal, and needs to be restarted regularly to maintain a certain level of active monitoring.  One of the way you could do it is with scheduled jobs.  But there are some drawbacks with this method.  The first one is that the configuration files are left unprotected, which can be problematic when it comes time to assuring data integrity of these files.  After all, what's the use of having modules to detect rogue users and rogue processes if any would-be attacker only has to modify these files to hide his presence?  Of course, the attacker would have to know that the tools are there, but to rely only on that is just plain security trough obscurity.  Another drawback is that you cannot have a granularity smaller than 1 minute at running the modules, which can be problematic if you want to apply special watch on critical systems.

For these reasons, LogAgent 5.0 Pro is equipped to handle the SIDTk modules when it is run as a service (licensed).  This is done so by LogAgent service stub program, logserv.exe (not to be confused with logservices.exe), who takes care of the necessary things needed to run as a service, and who can also take care of external modules by the means of the file modules.txt.

Here is a sample of modules.txt:

logservices,60,services.txt
logshares,60,shares.txt
logstartup,60,startup.txt
loguser,300,user.txt
logproc,20,processes.txt
adsscan,86400,
integcheck,43200,hash_conf.txt,hash_log.txt

This file is a 3-fields comma delimited text file, and the entries are built like this: the module name (with parameters if any), the timeout period (in seconds) at which the program is to be started, and a list of configuration files, separated by commas if more than one.  The reason to include the configuration files here is because logserv.exe will apply a filelock on them all the time the service is running, giving these files at least some protection (administrator level access is at least needed to terminate the programs).  The timeout period is specified in seconds, which allows for small granularity, while large timeout periods (like 12 or 24 hours) may be preferable for bigger processes like ADSScan and IntegCheck.  These modules will then be taken in charge by LogAgent, and will be run automatically in the background as services.

10. Conclusion

I have presented here the SécurIT Intrusion Detection Toolkit, otherwise known as the SIDTk, version 1.0.  I have explained the various reasons that lead me to separate this code from the previous commercial versions of LogAgent and LogIDS.  All of the drawbacks that have been identified with the previous generation of these tools have been fixed with this modular design.  In fact, this allows for the creation of more specialized modules, which can easily be included in the mix without redeploying the whole code base.

In fact, I made another module, LogSchTasks, which detects invalid scheduled tasks, but it has not been included in this release for stability reasons.  The program runs well when run from the command line, but when run as a service it eats up all the CPU available, and keeps growing in memory until the system has no resources left.  This is particular only to the fact that the program is running as a service, and if someone knows why I get this problem, I'd like to know.  I'd hint at some internal permission that could be missing to the account LogAgent is running under, or some unknown problem that causes the Perl function making the system call to recurse indefinitely.  Any help on this is appreciated.

Other than that, there may be other modules released in the future, it depends on the ideas I get.  If you have suggestions, or better yet if you want to create your own, I provide in Appendix A. a shell structure where you can start your program.  This shell provides all that is needed to read the config.txt file and send the output in a similar format.  If you create your own modules (using this code shell or not), let me know and maybe I'll include it in a future release of the SIDTk.

Appendix A. Code shell

# Code shell to create SITDk-standard modules
#########################################################################################
# by Floydman  floydian_99@yahoo.com							#
# Copyright 2003 SecurIT Informatique Inc.  http://securit.iquebec.com			#
#########################################################################################
## 
use Socket;
use Sys::Hostname;
use Win32::Lanman;
use Win32::Process::Info;

print "header goes here\n";

open(DATAFILE,"<file.txt") || die "Can't open file.txt";
@alloweddata = <DATAFILE>;
@alloweddata = parse (@alloweddata);
close (DATAFILE) || die "Can't close file.txt";

# Creation of machine ID table
@id = getid();

# Creation of config table
my @config = getconfig();

reportdata(); # this is the function where you put your analysis code

#########################################################################################
# procedure getconfig() 								#
# This procedure gets the configuration file config.txt.				#
#########################################################################################

sub getconfig
{
my @configtable;
my @dirtable;
$j = 0;
$numarg = 0;
open(CONFIGFILE,"<config.txt") || die "Can't open config.txt";
$logip = <CONFIGFILE> || die "Can't read  logip from config.txt";
($logip=~m/LOGIP/i) || die "LOGIP entry missing in config.txt";

$loghost = <CONFIGFILE> || die "Can't read loghost from config.txt";
($loghost=~m/LOGHOST/i) || die "LOGHOST entry missing in config.txt";

$loguser = <CONFIGFILE> || die "Can't read loguser from config.txt";
($loguser=~m/LOGUSER/i) || die "LOGUSER entry missing in config.txt";

$logdate = <CONFIGFILE> || die "Can't read logdate from config.txt";
($logdate=~m/LOGDATE/i) || die "LOGDATE entry missing in config.txt";

$logtime = <CONFIGFILE> || die "Can't read logtime from config.txt";
($logtime=~m/LOGTIME/i) || die "LOGTIME entry missing in config.txt";

$showconsole = <CONFIGFILE> || die "Can't showconsole read from config.txt";
($showconsole=~m/SHOWCONSOLE/i) || die "SHOWCONSOLE entry missing in config.txt";

$lp = <CONFIGFILE> || die "Can't read lineprint from config.txt";
($lp=~m/LINEPRINT/i) || die "LINEPRINT entry missing in config.txt";

$lpbuffer = <CONFIGFILE> || die "Can't read lineprint buffer from config.txt";
($lpbuffer=~m/LINEPRINT BUFFER/i) || die "LINEPRINT BUFFER entry missing in config.txt";

while (defined($dir = <CONFIGFILE>))
	{
	 $dirtable[$j]=$dir;
	 $j++;
	}
($j==0) && die "No destination directory specifed in config.txt.";
close (CONFIGFILE) || die "Can't close config.txt";

@configtable = ($logip, $loghost, $loguser, $logdate, $logtime, $showconsole, $lp, $lpbuffer, @dirtable);
@configtable = parse(@configtable);

(($numarg=@configtable)<9) && die "Not enough parameters in config.txt.  Check file for errors.";

# Tranformation of the first 7 lines of configtable to boolean value
$configtable[0]=$configtable[0]=~m/Y/i;
$configtable[1]=$configtable[1]=~m/Y/i;
$configtable[2]=$configtable[2]=~m/Y/i;
$configtable[3]=$configtable[3]=~m/Y/i;
$configtable[4]=$configtable[4]=~m/Y/i;
$configtable[5]=$configtable[5]=~m/Y/i;
$configtable[6]=$configtable[6]=~m/Y/i;

# Get the lineprint buffer
@element=split(/=/, $configtable[7]);
$configtable[7]=$element[1]; ($configtable[7]>=0) || die "LINEPRINT BUFFER entry must be 0 or a positive number in config.txt";
return (@configtable);
}

#########################################################################################
# procedure getid() 									#
# This procedure gets the IP address, the host name and the username of the machine.	#
#########################################################################################

sub getid
{
# Define username, IP address and hostname of the local machine
my $addr = inet_ntoa(scalar(gethostbyname($name)) || 'localhost');
my $host = hostname() || "hostname not defined";
my $login = getlogin || getpwuid($<) || "not logged";
my @id_table = ($addr, $host, $login);
return (@id_table);
}

#########################################################################################
# procedure parse(table_file) 								#
# This procedure cleans the files from non-valid and blank characters that could be	#
# placed in the config files.  The procedure returns the file as a table.		#
#########################################################################################

sub parse
{ my (@table) = @_;

#check for invalid characters in table_file
chomp @table;

foreach $element (@table)
 {
  $element=~s%^\s+%%;
  @char = split (//, $element);

  foreach $char (@char)
    { $char=~s%\\%/%; }
  $element = join ('',@char);
 }

my @tabletemp;
my $x = 0;

foreach $element (@table)
   {
    if ($element ne '') {
	$tabletemp[$x]=$element;
	$x++;}
   }
@table = @tabletemp;
return (@table);
}

#########################################################################################
# procedure reportdata()								#
#########################################################################################
sub reportprocesses
{
# This is where you collect information from the system in order to analyze it against @alloweddata
# you can make a foreach loop to check if the item you are analyzing is present in @alloweddata
# when you find an invalid item, simply use sendoutput with the proper parameters

# sendoutput takes as parameters the output filename, a string containing the comma-delimited
# log line, and the @config table.

##  Look at the SIDTk modules source code for working examples

# This is just a sample
@getdata = getsystemcoredata();

foreach $item (@getdata)
	{ my $ok = 0;
	 foreach $validdata (@alloweddata)
		{
		 if (lc($item->{'name'}) eq lc($validdate)) ($ok=1; last;)
		}
	 if (!$ok) {sendoutput ('filename.log',"Message goes here,".$item->{'name'}.","$item->{'info1'}.",".$item->{'info2'},@config);} #Don't forget the logs are comma-delimited
	}

}

#########################################################################################
# procedure sendoutput(filename, line, config)						#
# This procedure receives as arguments: the name of the modified file, the last line	#
# of the logfile, and then the config table (LOGIP, LOGHOST, LOGUSER, SHOWCONSOLE, and 	#
# the various destination directories).  The procedure checks the configuration to see	#
# if it has to append any information to the original line or not.  If SHOWCONSOLE in 	#
# enabled, then the line is printed on the screen, if not it simply passes to the next	#
# step which is to forward this line to all mentionned destinations in config.txt.	#
#########################################################################################

sub sendoutput
{ my ($filename, $lines, $logip, $loghost, $loguser, $logdate, $logtime, $showconsole, $lp, $lpbuffer, @dest) = @_;

my $line = '';
my @newlines = split (/\n/,$lines);
foreach $line (@newlines)
 {
 my $newline=""; 

 if ($logip) {$newline=$newline.$id[0].",";}
 if ($loghost) {$newline=$newline.$id[1].",";}
 if ($loguser) {$newline=$newline.$id[2].",";}
 if ($logdate || $logtime) {($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);} 
 if ($logdate) {$newline=$newline.($year+=1900)."/".($mon+1)."/".$mday.",";}
 if ($logtime) {$newline=$newline.$hour.":".$min.":".$sec.",";}

$newline=$newline.$line."\n";

  if ($showconsole) {print $newline;}

  if (lc($dest[0])ne"null") {
    foreach $destdir (@dest)
       {
        $destination=$destdir.$filename;
        open (DEST, ">>".$destination) || die "Can't open master log file $destination";
        flock (DEST, 2) || die "Can't lock file for writing";
        print DEST $newline || die "Can't write to file";
        close (DEST) || die "Can't close master log file";
       }
    }


  if ($lp) { push @buffer, $newline; $buffer=@buffer;
	   if ($buffer>=$lpbuffer+1) {
				     open(LINEPRINT,">LPT1") || die "Can't open pipe to LPT1";
				     foreach $line (@buffer){
					print LINEPRINT $line;}
				     close (LINEPRINT) || die "Can't close pipe to LPT1";
				     @buffer=[];
				   } 
         }
  
 }
}

