This chapter explores system services under DOS, Windows 3.x, Windows 95/98, and Windows NT. The authors discuss the need for hooking these system services.
THIS CHAPTER DISCUSSES hooking Windows NT system services. Before we begin, let’s first review what we mean by a system service. A system service refers to a set of functions (primitive or elaborate) provided by the operating system. Application programming interfaces (APIs) enable developers to call several system services, directly or indirectly. The operating system provides APIs in the form of a dynamic link library (DLL) or a static compiler library. These APIs are often based on system services provided by the operating system. Some of the API calls are directly based on a corresponding system service, and some depend on making multiple system service calls. Also, some of the API calls may not make any calls to system services. In short, you do not need a one-to-one mapping between API functions and system services. Figure 6-1 demonstrates this in context of Windows NT.
SYSTEM SERVICES: THE LONG VIEW
System services and the APIs calling these system services have come a long way from DOS to Windows NT.
System Services under DOS
Under DOS, system services comprise part of the MS-DOS kernel (including MSDOS.SYS and IO.SYS). These system services are available to users in the form of Interrupt Service Routines (ISRs). ISRs can be invoked by calling the appropriate interrupt handlers using the INT instruction. API functions, provided by compiler libraries, call the interrupt handler for system services (the INT 21h interrupt). For example, to open a file, MS-DOS provides a system service for which you have to specify the function number 0x3D in the AH register, attribute mask in the CL register, filename in the DS:DX register, as well as issue the INT 21h instruction. Compilers typically provide wrappers around this and provide a nice API function for this purpose.
System Services under Windows 3.x and Windows 95/98
Under Windows 3.x or Windows 95/98, the core system services take the form of VXDs and DLLs and some real-mode DOS code. The APIs are provided in the form of dynamic link libraries. These dynamic link libraries call the system services to implement the APIs. For example, to open a file, applications call an API function from KERNEL32.DLL such as OpenFile() or CreateFile(). These APIs, in turn, call a system service.
System Services under Windows NT
Under Windows NT, the NT executive (part of NTOSKRNL.EXE) provides core system services. These services are rather generic and primitive. Various APIs such as Win32, OS/2, and POSIX are provided in the form of DLLs. These APIs, in turn, call services provided by the NT executive. The name of the API function to call differs for users calling from different subsystems even though the same system service is invoked. For example, to open a file from the Win32 API, applications call CreateFile() and to open a file from the POSIX API, applications call the open() function. Both of these applications ultimately call the NtCreateFile() system service from the NT executive.
| Note: Under Windows NT 3.51, the system services are provided by a kernel-mode component called NTOSKRNL.EXE. Most of the KERNEL32.DLL calls—such as those related to memory management and kernel objects management—are handled by these system services. The USER32 and GDI32 calls are handled by a separate subsystem process called CSRSS. Starting with Windows NT 4.0, Microsoft moved most of the functionality of CSRSS into a kernel-mode driver called WIN32K.SYS. The functionality moved into WIN32K.SYS is made available to the applications in the form of system services. These system services are not truly part of native system services since they are specific to the user interface and not used by all subsystems. This chapter and the next chapter focus only on the system services provided by NTOSKRNL.EXE. |
NEED FOR HOOKING SYSTEM SERVICES
Hooking represents a very common mechanism of intercepting a particular section of executing code. Hooking provides a useful way of modifying the behavior of the operating system. Hooking can help the developer in several ways. Often developers are concerned more with how to hook a system service or an API call rather than why to hook. Nevertheless, we examine the various possible situations in which the need to hook a system service arises. How hooking can help the developer is explained in the following sections.
Trapping Events at Occurrence
Developers trap events such as the creation of a file (CreateFile()), creation of a mutex (CreateMutex()), or Registry accesses (RegCreateKey()) for specific purposes. Hooking a particular event-related API or system service call, synchronously, can help trap those events. Applications doing system monitoring will find these kinds of hooking invaluable. These hooks could act as interrupts triggered by the occurrence of these events. A developer could write a routine to handle the occurrence of these events and take appropriate action.
Modifying System Behavior to Suit User Needs
Diverting the normal flow of control by introducing the hooks can modify operating system behavior. This enables the developer to change data structures and context at the time of hooking–enough to induce new behavior. For example, you can protect the opening of a sensitive file by hooking the NtCreateFile() system service. Although NTFS provides user-level security for files, this security is not available on FAT partitions. You should ensure that hooking does not have any undesirable side effects on the operating system. Protecting modifications to Registry keys is something easily doable when you hook the Registry system services. This has several applications, since little protection is provided for Registry settings created by applications.
Studying the Behavior of the System
In order to get a better idea of the internal workings of the operating system, studying the behavior of the system is something most debuggers or system hackers will relate to. Understanding of undocumented operating system functionality requires a lot of hacking, which goes hand in hand with hooking.
Debugging
Complex programs could make use of system-service hooking to debug the stickiest problems. For example, a few days back, we had a problem with the installation of a piece of software. We had difficulty creating folders and shortcuts for this application. Using a systemwide hook, we quickly figured that the installation program was looking for a Registry value that indicated where to install the folders (which happened to be the Start menu). We hooked the NtQueryValueKey() call, then obtained the value the installation program was looking for. We created that value and solved our problem.
Getting Performance Data for Specific Tasks and Generating Statistics
These tasks can prove very useful to those writing benchmarks and applications to critically measure system performance under specific conditions. Even measuring the frequency of certain system services becomes very easy with this type of hooking. Measuring file system performance by hooking the file system-related system services exemplify this procedure.
Life without hooking is unthinkable for most Windows developers in today’s Microsoft-dominated world of operating systems. Windows NT system services lie at the center of the NT universe, and having the ability to hook these can prove extremely handy.
TYPES OF HOOKS
The following sections explore two types of hooking.
Kernel-Level Hooking
You can achieve kernel-level hooking by writing a VXD or device driver. In this method, essential functions provided by the kernel are hooked. The advantage of this type of hooking is that you get one central place from which you can monitor the events occurring as a result of a user-mode call or a kernel-mode call. The disadvantage of this method is that you need to decipher the parameters of the call passed from kernel mode, since many times these services are undocumented. Also, the data passed to the kernel-mode call might differ from the data passed in a user-mode call. Also, a user-level API call might be implemented using multiple calls to the kernel. In this case, hooking becomes far more difficult. In general, this type of hooking is more difficult to achieve, but it can produce more rewarding results.
User-Level Hooking
You can perform this type of hooking with some help from a VXD or device driver. In this method, the functions provided by the user-mode DLLs are hooked. The advantage of this method is that these functions are usually well documented. Therefore, you know the parameters to expect. This makes it easy to write the hook function. This type of hooking limits your field of vision to user mode only and does not extend to kernel mode.
IMPLEMENTATIONS OF HOOKS
The following sections detail the implementation of hooks under various Microsoft platforms.
DOS
In the DOS world, system services are implemented as an interrupt handler routine (INT 21h). The compiler library routines typically call this interrupt handler to provide an API function to the programmer. It is trivial to hook this handler using the GetVect (INT 21h, AX=25h) and SetVect (Int 21h, AX=35h) services. Hence, hooking system services are fairly straightforward. DOS does not contain separate user and kernel modes.
Windows 3.x
In the Windows 3.x world, system services are implemented in DLLs. The compiler library routines represent stubs that jump to the DLL code (this is called dynamic linking of DLLs). Also, because the address space is common to all applications, hooking amounts to getting the address of that particular system service and changing a few bytes at that address. Changing of these bytes sometimes requires the simple aliasing of selectors.
| XREF: Refer to the MSDN article in Microsoft Systems Journal (Vol. 9, No. 1) entitled, “Hook and Monitor Any 16-bit Windows(tm) Function With Our ProcHook DLL,” by James Finnegan. |
Windows 95 and 98
In the Windows 95/98 world, system services are implemented in a DLL as in Windows 3.1. However, under Windows 95/98, all 32-bit applications run in separate address spaces. Because of this, you cannot easily hook any unshared DLL. It is fairly easy to hook a shared DLL such as KERNEL32.DLL. You simply modify a few code bytes at the start of the system service you want to hook and write your hook function in a DLL that is loaded in shared memory. Modifying the code bytes may involve writing a VXD, because KERNEL32.DLL is loaded in the upper 2GB of the address space and protected by the operating system.
Windows NT
In the Windows NT world, system services are implemented in the kernel component of NT (NTOSKRNL.EXE). The APIs supported by various subsystems (Win32, OS/2, and POSIX) are implemented by using these system services. There is no documented way of hooking these system services from kernel mode. There are several documented ways for hooking user-level API calls.
| XREF: Refer to the MSDN
articles in Microsoft Systems Journal
entitled, “Learn System-Level Win32(r) Coding
Techniques by Writing and API Spy Program,” by
Matt Pietrek (Vol.9, No.12), and “Load Your
32-bit DLL into Another Process’s Address Space
Using INJLIB,” by Jeffrey Richter (Vol.9,
No.5). Refer to CyberSensor on http://www.cybermedia.co.in |
We will present one way of achieving hooking of NT system services in kernel mode in this chapter. We also provide the code for this on the CD-ROM accompanying this book.
Page: 1, 2
next page
