While I managed to figure out most of this stuff myself, I highly reccomend System Internals to anyone who is looking for a good source for code and documentation about low level Win32 programming. Their regmon and filemon are both great open-source tools, and largely duplicate the functionality of my own system monitor (which essentially does what both of these do in one program). Full creds where they are due, you guys rock!
The current method that most AV companies use to protect against hostile code is signature scanning. They compile a list of 'signatures' for each piece of hostile code that has been discovered and scan each executable as it is run. The signatures are a group of bytes that have been found in the hostile code that have been determined to be 'unique' to the hostile code. AV software might also do additional checks to make these signatures are apearing in the correct part of the file to prevent false reports, because mathmatically speaking, with 9 gigs of data on your hard drive there is a fairly good chance that one of the hundreds of signatures being scanned for will occur somewhere in some data on your drive.
Signature scanning is a reasonable defense against self-replicating viruses because they tend to be 'turned loose' and do their damage. Of course, all it takes is for someone to modify a known virus so that the signature being scanned for is changed, and until the new 'strain' has been discovered, a new signature extracted, and the new signatures distributed to all the users of the AV software, users have no protection.
However, there are several problems with signature scanning, the most obvious of which is that it only protects users against KNOWN HOSTILE CODE. All it takes to defeat signature scanners is to make new hostile code. A slightly less obvious problem, but much more frightening imho, is that signature scanners only scan for signatures in the data as it exists on disk. Once a program is loaded into memory and executed, there is nothing to stop it from creating whatever code it wants in memory and running it. If it choses to put hostile code in memory and execute it, signature scanners will not even notice if the hostile code being executed contains the signature of a known problem. Even using a commercially available executable compresser is enough to hide as well known a program as Back Orifice. I've even seen a program called PE Sentry which does nothing more than rotate every byte in the original executable one bit, as soon as the program runs, it rotates all the bytes back to the original and executes the original code. You can try this yourself, download the demo of Shrinker or find a program like PE Sentry, take your favorite scanned-for trojan (like BO or whatever), and compress/ encrypt it. Now scan it. Now run it. Any problems?
One note on configuring a compressed/encrypted trojan: because the executable must cary all its configuration information around with it, it must imbed it in the executable somehow. The issue is simply weather you should compress the exectuable before or after it's been encrypted/compressed. BO appends the configuration to the end of the executable, so you must configure it AFTER it has been compressed. BO2K puts its configuration information into one of the actual sections of the exe, so you must configure the exe BEFORE it is compressed. Other programs, I have no clue about.
This all means that signature scanning protects you from only the lowest anonymous form of attack. If someone were to target YOU specifically, you have absolutely no defense. In fact as the average script kiddie becomes aware of these issues (and believe me, they know) signature scanning becomes even less effective against these low forms of attack. I've talked to dozens of people who were motivated to learn to program just so they could write their own remote control trojan application.
Signature scanning is not the only know method of defense against hostile code. The idea of monitoring the actual run time operations of an application is nothing new, but it seems that few people have even attempted it since we were running CPUs too slow to devote resources to this method of defense. With CPU power increasing exponentially and the cost comming down daily, users should at least have the option of sacrificing CPU time for extra security.
I would like to note that while I was unaware that anyone had actually attempted implementing functional application sandboxing, since Defcon 7 I have been contacted by one AV company who says they are already working on such a product and another who apparently already has a product which accomplishes this. I have not personally seen either piece of software, but if they do what they claim, I don't understand why nobody I've talked to has even heard of these products.
The idea behind application sandboxing is very straigt-forward: run a possible hostile application in a controlled enviroment so that if a dangerous operation is attemted, it can be intercepted before any damage occurs. The technical details of implementing this requires knowledge of ring 0 (VxD) programming, but is obviously not impossible. They key to implementation is going to be keeping the machine usable.
Obviously no method of defense is going to be %100 bulletproof, especially if any judgements are left to the user, but at least using a method other than signature scanning raises the level of technical expertise necessary to defeat the current means of defense from ANY programmer who has the time to write something new to a SKILLED programmer who has discovered a specific hole in security. At least this method makes it an issue of exploits and fixes, rather than chasing every script and bomb written by every VB programmer.
First off, it is important to remember that am NOT paid to do this stuff, this is what I do for FUN in my SPARE time. I also had to teach myself almost all of the methods involed just so I could accomplish the idea I had for a stronger method of protection. My method is also specific to Win9x (because that is my current area of focus) but the same concepts should be able to be applied to any operating system that has the means to hook resource access.
Using a VxD, a program can hook all access to any resources available made by any application or part of the operating system. Currently, I hook the file system using IFSMGR_InstallFileSystemApiHook and the registry using Hook_Device_Service (hooking each of the registry APIs). This causes the system to call functions I have written whenever any application accesses any of the file or registry APIs BEFORE they are actually sent to the driver layer to be processed. This allows me to examine what operation is being attempted with what access, determine if the process attempting this operation has the access required for this operation, and if not presenting the user with the ability to allow or not allow the operation, or terminating the application.
As a first attempt, I have written a program which allows you to execute a single application and monitor and control the actions of that application. Spawning that application as a 'debugged' application, I am also to hook when that application attempts to spawn other applications at ring 3, which are also monitored and controlled. Before the original application is spawned, the user is allowed to specify the level to which it wants to control this program; full (any reads and writes are hooked), read only (all reads are allowed, only potential write operations are hooked) and monitor only (allow the application to smash your system if it wants, but log what it does). Any time a registry or file access is attempted that exceeds the programs permission, the VxD halts the system with a blue screen, presents the user with the information about what application (remember, the original program may spawn additional apps) is attempting what access (read/write) on what resource (file/key/value). If the user allows the operation, the application will be allowed to perform that operation in the future (in other words, you don't have to keep hitting OK every time the program tries to access its registry keys or whatever) and the system resumes. After the original application and any it spawned has terminated, my program presents a log of all reads and writes and program spawns that were attempted.
Another application I have developed simply logs all registry and file access system wide. One difference between my application and most others I have seen is that rather than just logging the 'module name' (the 8 character short name), I attempt to retrieve the full long filename path to the executable. This turns out to by fairly difficult. Because in a Win32 enviroment you may have Win32 exe's, Win16 exe's, and DOS exes, there is a fair amount of logic involved in finding the location of the full path, and it involves knowledge of several undocumented internal Windows structures and makes assumptions that things will be in memory where they happen to be. Check here for more info on the internal structures.
char * VXD_GetProcessName()
{
struct cb_s *cbptr;
WORD w;
char *ptr;
PDWORD pdw;
PPDB pdbptr;
PPSP pspptr;
LPTDB tdbptr;
LPMODULE modptr;
POFSTRUCT ofptr;
char *nameptr;
DWORD dw;
char buff[512];
// g_modulelist is a pointer to a PIMTE array which is passed from ring 3
if (g_modulelist == NULL)
return "g_modulelist == 0";
// get current virtual machine handle
cbptr = (struct cb_s *) Get_Cur_VM_Handle();
if (cbptr->CB_VM_Status & VMSTAT_PM_APP) // process is a win16/win32 app
{
if (Get_Cur_Thread_Handle() == VXD_Get_Sys_Thread_Handle())
{
// if it's the system performing the operation...
nameptr = "(System)";
} else {
// in ring 0, a process handle is actually a pointer to the PDB structure in memory
pdbptr = (PPDB)VXD_GetCurrentProcessHandle();
if (pdbptr == 0)
{
nameptr = "Unable to retrieve current process handle";
} else {
if (VXD_IsClientWin32()) // if it's a win32 app
{
if (pdbptr->MTEIndex != -1 && g_modulelist[pdbptr->MTEIndex] != NULL)
{
if (_Assert_Range(g_modulelist[pdbptr->MTEIndex], sizeof(IMTE), -1, 8, 0) != 0)
{
if (g_modulelist[pdbptr->MTEIndex]->pszFileName != NULL)
{
nameptr = g_modulelist[pdbptr->MTEIndex]->pszFileName;
} else {
nameptr = "pszFilename = NULL";
}
} else {
_Sprintf(buff, "Bad range MTEIndex = 0x%x imte[MTEIndex] = 0x%08x", pdbptr->MTEIndex, g_modulelist[pdbptr->MTEIndex]);
nameptr = buff;
}
} else {
if (pdbptr->MTEIndex == -1)
nameptr = "MTEIndex == -1";
else
nameptr = "modlist[MTEIndex] == 0";
}
} else { // win16 app
if (pdbptr->W16TDB)
{
// retrieve flat pointer to win16 thread database structure
dw = pdbptr->W16TDB;
_asm mov eax, dword ptr[dw] ;
tdbptr = (LPTDB)VXD_SelectorMapFlat();
if (tdbptr != (LPTDB)0xffffffff)
{
if (tdbptr->TDB_sig = 0x4454) //'TD'
{
w = tdbptr->TDB_HMODULE;
_asm mov ax, word ptr[w] ;
modptr = (LPMODULE)VXD_SelectorMapFlat();
if (modptr != (LPMODULE)-1)
{
if (modptr->ne_signature == 0x454e) // 'NE'
{
if (modptr->ne_npFileInfo)
{
ofptr = (POFSTRUCT)(((char *)modptr) + modptr->ne_npFileInfo);
nameptr = ofptr->szPathName;
} else {
nameptr = "ne_npFileInfo = 0";
}
} else {
nameptr = "bad NE signature";
}
} else {
nameptr = "Unable to map module";
}
} else {
nameptr = "bad TDB";
}
} else {
nameptr = "unable to map tdb";
}
} else {
nameptr = "w16TDBSEL = 0";
}
}
}
}
} else { // else DOS app
pspptr = (PPSP)VXD_GetDOSPSP();
dw = pspptr->wEnvSeg;
nameptr = (char *)(dw << 4);
do
{
nameptr+= _lstrlen(nameptr) + 1;
} while (*nameptr != 0);
nameptr+=3;
if (nameptr[1] != ':') // if it's not full path
{
nameptr = (char *)VXD_GetDOSFileName();
}
}
return nameptr;
}
The only weak link in this method is for certain DOS executables, usually .COM files. I think this
might be because I am retreiving the full DOS path name based on the undocumented assumption that
this info ends up being in memory right after the enviroment variables, but this might be something
that is set up by the command interpreter, so for command interpreters themselves (command.com,
4dos.com etc) it isn't always there.
Another thing that concerns me about this method is that there really isn't anything to prevent an application from writting their own info to these structures in memory...
The product I would like to see available eventually is a system wide application monitoring system. Every program running will have a list of the resources it is allowed to access, every program would be monitored, and any time an application attampted to access a resource it never had before, the user would have to explicitly allow that application additional access. Combine this with a simple password to change access, and you even have local machine security (someone can't just walk up, stick in a floppy and run whatever they want). This would protect against not just known and unknown 'hostile' code, but also any bugs that might be in software you are already running and trust. Remember the Netscape client bug that allowed the server to download the files of its choice off the client machine? As soon as netscape.exe tried to access a file it never had before, alarm. I would also like to be able to hook other resources, most importantly network resources. It would be possible for a program to for instance hook the keyboard, open a network connection, and tramsmit the keystrokes without ever setting any allarms off.
More to come...