grimani
August 4th, 2005, 21:50
a quick little FAQ for those like me who can't seem to get softice to work and are too scared of mods to ask for help.
disclaimer: i tried a million things to get softice to work. it works now on XP SP2 with all the latest updates (i think), and i think the following steps are why...but accuracy is NOT guaranteed.
(i) what's this API hook failed thing?
softice needs to insert itself in between all system level API calls in order to work. hence it needs to know where in memory critical functions (ntterminateprocess, reside. i guess this is typically stored in the osinfo.dat and osinfob.dat files. unfortunately, yours are out of date.
(ii) why is it out of date?
in my case, i installed security update KB890859 which includes a new version of the kernel, presumably to close holes in the old one.
whenever a new version of the kernel (which lives at c:\windows\system32\ntoskrnl.exe) is released, addresses get all shuffled and new versions of osinfo.dat/osinfob.dat need to be used. since compuware is lazy, you're out of luck.
thankfully, i found a compuware document via google the describes another solution.
(iii) symbols
function names and the like are usually stripped from executables during the compiling process, if they are meant for public consumption. if code is still in development, debug executables are compiled which keep all this information intact.
to debug retail (public) executables, one can generate "symbols" that debuggers then load. symbols are needed by the debugger to figure out what function is what, what parameters it takes, where it is, etc.
so softice, being a debugger, has functionality to load symbols - it's necessary in debugging.
and why isn't softice working? it doesn't know where important functions live...!
clearly the solution is to load symbols for all the important API calls. these reside in a variety of files, including:
hal.dll
ntoskrnl.exe
ntdll.dll
hernel32.dll
user32.dll
csrsrv.dll
basesrv.dll
winsrv.dll
we want to get symbols for them. but microsoft developed/compiled these files, not us..
(iv) downloading symbols with symbol retriever
so we need to get symbols from microsoft. microsoft provides a DDK that apparently contains a pretty comprehensive set of symbols for their code. however, my suspicion is that the DDK is out of date or applies only for vanilla installs of the operating system. so that's out of the question.
alternately, microsoft runs a symbol server (never knew they did that!) thru which 'authorized' programs can download symbols. authorized in this case i think means the symbol retriever that comes with windbg, microsoft's debugging tool.
softice also has a symbol retriever. you can try to run it on the above files. it doesn't work. why? because, as with all things softice, it's out of date and symbol server doesn't want to play ball.
a very helpful post somewhere (here? exetools? i forget) notes though that one can copy the symsrv.dll file from the windbg distribution into the symbolretriever directory in softice and overwrite the old symsrv.dll that softice has.
voila! symbol retriever works.
(v) generating nms files
microsoft debugging information is stored in .dbg or .pdb files. softice, for reasons unknown, uses .nms files. i assume that stands for numega symbols.
but symbol retriever can convert .dbg to .nms. have it do that.
(vi) loading symbols
almost done. we still gotta tell softice to load those damn symbols. edit c:\windows\system32\drivers\winice.dat
that file contains most of the settings that softice looks at while loading. add a new line at the top:
NTSYMBOLS=ON
this will tell softice to use symbol files instead of osinfo.dat.
look to the INIT= line. remove the X;
INIT contains the commands the softice will run after starting up. X means stop debugging and let everything run. if softice don't work you want to see the errors, which means staying in softice. you can put the X; back in later if you'd like.
save the file, now go into the settings program and click on symbols. add all the nms files you generated.
save everything, close all unnecessary programs. now try running softice:
start|run|cmd.exe
net start ntice
cross your fingers and hope for the best. remember that, if even scrushy can be innocent, perhaps there is a god, however blind He may be, after all......
other stuff
(vii) firewalls? antivirus?
firewalls and antivirus programs frequently also hook into the API. certain spying programs that people install to covertly monitor computer usage may also do the same thing.
these things may conflict with softice. i uninstalled norton systemworks (it's a slow piece of shit anyway that takes over the computer) in the middle of my quest to get softice working.
was that necessary? dunno...but softice works now and i'm not about to jeopardize that.
speaking of norton, systemworks has tentacles everywhere, and the uninstaller is a little too ill conceived to kill all of them. search on the norton website for some utilities (they have 4!) that can remove all traces of the beast. i think i had to run all 4...which is a pain, but much better than trouncing through the registry on my own.
(viii) usb, mice, keyboards, and other random woes.
just to make things easy on yourself, disable the mouse just to keep things simple. i have a laptop with an internal usb wifi card. i turned that off too.
if you get softice to work without usb support then you can look into getting usb devices working.
lotta people have keyboard issues, but thankfully i have had none.
finally, i installed softice on my laptop. the touchpad doesn't work. probably a driver issue. it probably won't work on yours either. synaptics has some document somewhere that mentions softice specifically.
(ix) video issues
had none, so won't comment. just try universal video driver
disclaimer: i tried a million things to get softice to work. it works now on XP SP2 with all the latest updates (i think), and i think the following steps are why...but accuracy is NOT guaranteed.
(i) what's this API hook failed thing?
softice needs to insert itself in between all system level API calls in order to work. hence it needs to know where in memory critical functions (ntterminateprocess, reside. i guess this is typically stored in the osinfo.dat and osinfob.dat files. unfortunately, yours are out of date.
(ii) why is it out of date?
in my case, i installed security update KB890859 which includes a new version of the kernel, presumably to close holes in the old one.
whenever a new version of the kernel (which lives at c:\windows\system32\ntoskrnl.exe) is released, addresses get all shuffled and new versions of osinfo.dat/osinfob.dat need to be used. since compuware is lazy, you're out of luck.
thankfully, i found a compuware document via google the describes another solution.
(iii) symbols
function names and the like are usually stripped from executables during the compiling process, if they are meant for public consumption. if code is still in development, debug executables are compiled which keep all this information intact.
to debug retail (public) executables, one can generate "symbols" that debuggers then load. symbols are needed by the debugger to figure out what function is what, what parameters it takes, where it is, etc.
so softice, being a debugger, has functionality to load symbols - it's necessary in debugging.
and why isn't softice working? it doesn't know where important functions live...!
clearly the solution is to load symbols for all the important API calls. these reside in a variety of files, including:
hal.dll
ntoskrnl.exe
ntdll.dll
hernel32.dll
user32.dll
csrsrv.dll
basesrv.dll
winsrv.dll
we want to get symbols for them. but microsoft developed/compiled these files, not us..
(iv) downloading symbols with symbol retriever
so we need to get symbols from microsoft. microsoft provides a DDK that apparently contains a pretty comprehensive set of symbols for their code. however, my suspicion is that the DDK is out of date or applies only for vanilla installs of the operating system. so that's out of the question.
alternately, microsoft runs a symbol server (never knew they did that!) thru which 'authorized' programs can download symbols. authorized in this case i think means the symbol retriever that comes with windbg, microsoft's debugging tool.
softice also has a symbol retriever. you can try to run it on the above files. it doesn't work. why? because, as with all things softice, it's out of date and symbol server doesn't want to play ball.
a very helpful post somewhere (here? exetools? i forget) notes though that one can copy the symsrv.dll file from the windbg distribution into the symbolretriever directory in softice and overwrite the old symsrv.dll that softice has.
voila! symbol retriever works.
(v) generating nms files
microsoft debugging information is stored in .dbg or .pdb files. softice, for reasons unknown, uses .nms files. i assume that stands for numega symbols.
but symbol retriever can convert .dbg to .nms. have it do that.
(vi) loading symbols
almost done. we still gotta tell softice to load those damn symbols. edit c:\windows\system32\drivers\winice.dat
that file contains most of the settings that softice looks at while loading. add a new line at the top:
NTSYMBOLS=ON
this will tell softice to use symbol files instead of osinfo.dat.
look to the INIT= line. remove the X;
INIT contains the commands the softice will run after starting up. X means stop debugging and let everything run. if softice don't work you want to see the errors, which means staying in softice. you can put the X; back in later if you'd like.
save the file, now go into the settings program and click on symbols. add all the nms files you generated.
save everything, close all unnecessary programs. now try running softice:
start|run|cmd.exe
net start ntice
cross your fingers and hope for the best. remember that, if even scrushy can be innocent, perhaps there is a god, however blind He may be, after all......
other stuff
(vii) firewalls? antivirus?
firewalls and antivirus programs frequently also hook into the API. certain spying programs that people install to covertly monitor computer usage may also do the same thing.
these things may conflict with softice. i uninstalled norton systemworks (it's a slow piece of shit anyway that takes over the computer) in the middle of my quest to get softice working.
was that necessary? dunno...but softice works now and i'm not about to jeopardize that.
speaking of norton, systemworks has tentacles everywhere, and the uninstaller is a little too ill conceived to kill all of them. search on the norton website for some utilities (they have 4!) that can remove all traces of the beast. i think i had to run all 4...which is a pain, but much better than trouncing through the registry on my own.
(viii) usb, mice, keyboards, and other random woes.
just to make things easy on yourself, disable the mouse just to keep things simple. i have a laptop with an internal usb wifi card. i turned that off too.
if you get softice to work without usb support then you can look into getting usb devices working.
lotta people have keyboard issues, but thankfully i have had none.
finally, i installed softice on my laptop. the touchpad doesn't work. probably a driver issue. it probably won't work on yours either. synaptics has some document somewhere that mentions softice specifically.
(ix) video issues
had none, so won't comment. just try universal video driver