What is the purpose of the instructions repz ret used together? I have done a bit of research and it seems to be related to a difference in the way an AMD and Intel processor handles instruction, but I am wondering if it is used in some way for protection?

It follows a JNZ instruction where the Z flag is set. So the code looks as follows:

JNZ address
REPZ RET
JMP address

The machine code is:

7502
F3C3
E9 (address)

I did a search on RCE and saw a reference to REPZ NOP, I think as a means to cause a sleep situation. I am not sure how that would work.

http://repzret.org/p/repzret/#more-1

naides...thanks for link. I read one that was similar but not as well explained as this one. Logically, I wondered what would happen with the repz instruction if the zero flag was never toggled but I guess without arguments it just acts like a NOP. I wonder what the logic for the statement looks like in the CPU.

I wonder if you might have an explanation for another code issue. An idiv instruction came up while tracing code in a winproc. It is comparing atoms, which I have yet to identify. One technique it uses that is not clear to me is dividing the value of an hwnd like 160274 by 64 and getting a value in eax and the remainder in edx. It uses the remainder to point into an index using something like [eax*4+ecx], where eax is the remainder from the idiv operation, and finds a dword there it compares to 0. It might be looking for the end of a list. The address range pointed to seems far out of the apps image space in memory but could be a heap space. Often, the addresses pointed to point themselves to RDATA values.

I would imagine it is trying to identify an identifier related to a window, such as a button or a text string, ultimately to find the code that deals with that identifier. It uses different techniques to narrow in on the identifier like placing it between certain values, but then it seems to adopt the technique described. I am wondering what the window handle has to do with narrowing a search for a value. In the fog in my brain, I think the handle is a selector that is made up of different parts but I don't see the relationship between a handle for a pop up window with an edit box, text strings and buttons and dividing it by 64.

looks like the internal WND structure can be found from a HWND

http://winterdom.com/dev/ui/wnd
http://cyberkinetica.homeunix.net/os2tk45/sg244640/209_L5_FindingaWNDFromanHWN.html

Thanks for the links. The second one is for the OS2 system which was abandoned a long time ago, so you have to be careful with that. The older win32 versions (eg. win95) had a 16 bit base using user and krnl, which were 16 bit apps. User (not user32) handled windows. In those, user could interface with 32 bit process using a lookup table in which the hwnd produced by user was a pointer into a table that referenced 32 bit addresses storing window information.

I am not sure where things stand with XP and Win 7 code but I had a hwnd for an edit window recently with a hwnd value of 160xxx, which has 24 bits. I don't think user (16 bit) could have handled that, so it seems later versions of win32 are using a different scheme.

However, I traced my 24 bit hwnd through the dark codewoods of system code using softice and followed it to @validatehwnd in win32k. I saw it strip off the LSBits of hwnd and process that to get a pointer into a table. In one of Matt Pietreks articles from the 90s, he mentioned that user32 processes hwnds using a dispatchmessage equivalent and sends the hwnd to win32k, which he claims is the system version of U32. He mentioned that win32k uses @validatehwnd to check for a legitimate window.

It seems, therefore, that a hwnd can be parsed to give a pointer into a table, so it makes sense that dividing it by 64, and so on, does something to find which window is being processed, or maybe a child window. I don't know if hwnds for child windows have a relationship to the hwnd of the parent window.