Whenever I try to bring up the handles window - in Ollydbg 1.10 and Immunity Debugger both - it stops responding until I kill the debugee. Is there something obvious I'm overlooking as to why this would happen?

ollydbg uses some undocumented api functions to get handle info. are you using any funky av/firewall software? if so, it may be messing with olly's low-level api calls.

I have AVG running, and Comodo firewall. It'd seize up before when running Comodo firewall alone.

Tried killing those services manually (this is XP SP2, by the way), but still no luck.

If it's of any use, after the debugee is killed, the handles window does refresh with information.

I does not happen in my system, so it likely has to do with something with either your Olly, your system or your application(or your karma:devil.
By systematically ruling out each possibility you can narrow down the source of the problem:

Problem solving 101:

-Does it happen with every app you debug or only a particular instance?

- Is it the AV or the firewall? Not only kill the services but uninstall (temporarily) all of them, then check if the problem persists.

-Is it Olly? Run a fresh virgin copy without any Plug-ins and see if the problem repeats itself.

I think, you should test a virtual machine!?

If you have NVIDIA based motherboard check if you have installed NVIDIA Firewall/Network Access Manager and uninstall that too.