Does any one know how does the SETxx family of opcodes select their memory operands?
They are the only ones that take single operand, and a ModRegRm. The operand is m8 or r8. The register operand is retreived by looking up the value of the Reg field, in the array of one byte registers.

I have assembled a long list of varying SETxx instructions, both with register and memory operands. And I noticed that all the opcodes that take a byte of memory as an operand, are followed by a 6 (00000110).

The Mod is 00, meaning, no displacement, and the Reg is 000 meaning ESP in 32-bit. Now that is not what i coded.

If the R/M field was a 100, i would expect one more byte for SIB, but R/M is 110 !

I expect the offset of the operand to be encoded in the instruction, and indeed, I see the offset in the instruction. But when automating the disassembly, how do I differentiate between the two distinct operands. Should I stick to the 0x6 i have discovered, so when ever i parse a 6, i flag it as a memory operand, or is there a `traditional' way?

So far, i am confident about this magical 6. I have greped 30mbs worth of disassembly listings, and got the six with every memory operand for a SETxx (e.g. 0F 94 06 00 30). Where the operand is a register, only a single ModRegRm byte is given (e.g. 0F 94 0C), in the later case, only the Reg part is decoded, Mod and R/M remain intact!

Anyone has an idea?

Hi Mo,

well, I would like to know which disassembler you are using for this analysis. Certainly in a 32-bit segment you are not getting the disassembly right.

0f94c1 = sete cl
for example
now
0f9406 = sete byte ptr [esi]
and it is 05 which is the straight disp32, and 04 which indicates a sib
so
0f940542414000 = sete byte ptr [404142]

I checked this in Borg, and rechecked it in ida, setting the segment to 32-bit in a bin file.

Look who is here, I knew i would fetch a legend out :-)

What i posted was crap, i said SETxx where the only single operanded opcodes that take a ModRegR/M byte, not so, look at group 4/5/6

Well, I am playing with a prototype disassembler at the moment, to be writen from scratch. And I thought the 0F9x where special, and such, wanted to code their own ModRm decoder, and dedicate the big ModRm decoder for the opcodes with two or more operands.

Its quite irritating in the case of the single operanded instructions. I am deciding the src/dest combination by testing the second bit of the opcode bit-1. Where the opcode takes two operands, this is Ok. Three operands? still ok, the third operand is (mostly?) either 1 or CL, and i can pretty much ignore it. I cann't base my decision of the src/dest by the misleading comma in the opcode table, it was meant as a reading aid, not disassembly.

When an opcode takes a single operand, there is no need to find src or dest. I think i should handle this seperatly, either by flagging them as such in the opcode tables, and testing for this in every invocation of the ModRm decoder. Or, by writing a mini ModRm decoder for them, which is well tailored for The SETxx and groups 4/5/6, and handle them there.

Well, if you are wondering why this awkwardness, well, i want the disassembled instruction to be output to a structure which is then passed to the printing module. I want to extract more information from a disassembled instructions (i.e. cross-referencing, predicting disassembly, etc.) and such, would like further inspection. Syntax independance is also a requirement of mine. Off-course, this goals could be adjusted, depending on mood and time ;-)

As you can see, I am still working on the blue print, so, I have nothing -yet- to bring out to public.

Well, that is all, Just stick around for tech support :P

hehe!

Presumably you are realising and using the fact that the 'modr/m' byte often determines the actual instruction too (like SAL/SAR).