Log in

View Full Version : I'm little confused between VirtualSize and SizeOfRawData

Mo7attem
May 10th, 2007, 04:22
Hi all

I'm confused between the two members in the SECTION HEADER, the VirtualSize, and the SizeOfRawData, I've read a lot of tutorials about PE files, and as I understood: the VirtualSize is the total size of the section "in memory", and the SizeOfRawData is the total size of the section on the "Hard Disk" , I've also read that the SizeOfRawData may be greater than the VirtualSize, this because the linkers (or compliers, I forgot ) are tend to round the section up to the "FileAlignment", and in this case the rest of the size in the SizeOfRawData is filled with "Zeros", at this point I have no problem ..

but Here I get confused: while I was checking some PE files, I found some of them have VirtualSize GREATER than SizeOfRawData !!! How this can be ?!

as I understood the VirtualSize in fact is a part of SizeOfRawData, but how can it be greater than it ? for example: if the SizeOfRawData is 500byte , the PE loader will load the VirtualSize in Memory, which may be all of SizeOfRawData, or it may be some of it (400byte for example).
but how can The PE loader load the VirtualSize if it size is "say" 700byte ?! where will the "extra" 200bytes come from ?!
the total size of the section is only 500byte in the desk! how to load 700 in memory :|

I don't know where I have the misunderstood,
I hope you've understood my confusion, and please help me!

btw: I have a satisfactory (but little) experience with PE files, I know how change EP, how to add code, how to crypt certain sections, how to convert RVA or (VA) to Offset and vice-versa, and I've read about 4 or 5 good tutorials about PE files, but I didn't find the answer of my question :|
Nukacola
May 10th, 2007, 04:57
Quote:
I understood: the VirtualSize is the total size of the section "in memory", and the SizeOfRawData is the total size of the section on the "Hard Disk" , I've also read that the SizeOfRawData may be greater than the VirtualSize, this because the linkers (or compliers, I forgot ) are tend to round the section up to the "FileAlignment", and in this case the rest of the size in the SizeOfRawData is filled with "Zeros", at this point I have no problem

I think that is wrong cos u have switched SizeOfRawData and VirtualSize if u switch these two i agree with that.
Mo7attem
May 10th, 2007, 05:05
Quote:
[Originally Posted by Nukacola;65557]I think that is wrong cos u have switched SizeOfRawData and VirtualSize if u switch these two i agree with that.


Hi Nukacola, Are you sure about you've said ???

Isn't SizeOfRawData is the size of the section on the disk ??? I read this in many toturials !

I think you need to look up about this.

Please anyone give me a certain and accurate answer
Nukacola
May 10th, 2007, 05:11
that's not what i mean. I wanna say if the VirtualSize is greater than the SizeOfRawData the PE-loader is filling the rest with zeros. I don't know if it is possible that SizeOfRawData is greater than VirtualSize, but if so i think the linker will load only the data which fit.

And of of Course
SizeOfRawData = size on disk
VirtualSize = size in memory
i complete agree with you but what i try to say in my first post is if u switch the position of the both words the sentence is correct and answer your question

Greetz Nukacola
Mo7attem
May 10th, 2007, 05:20
Quote:
[Originally Posted by Nukacola;65559]that's not what i mean. I wanna say if the VirtualSize is greater than the SizeOfRawData the linker is filling the rest with zeros. I don't know if it is possible that SizeOfRawData is greater than VirtualSize, but if so i think the linker will load only the data which fit.

Greetz Nukacola


Hi Nukacola again, from what you wrote, I can tell (not sure, though) that you aren't professional in PE files, because in my little experience, I've found most of the sections in PE files have SizeOfRawData is greater than VirtualSize, and I've examined those files in HexEditors and I've found the rest of the SizeOfRawData (when it is greater than VirtualSize) is filled with zeroes.

EDIT:
OK, now I read your post after the edit
so, I'm still asking my question, while the SizeOfRawData is the Size of the section on the disk, and the VirtualSize is the size of the section in memory, from where the PE loader load the extra size (if the VirtualSize is greater than SizeOfRawData) ???
if the virtualsize is 700 and the SizeofRawData is 500, from where the PE loader will load the extra 200 bytes !!! :'(

I'm still confused :'(

Please anyone give me a certain and accurate answer
Nukacola
May 10th, 2007, 05:29
filled with zeros
Mo7attem
May 10th, 2007, 06:11
Aha! thank you for the answer !
so if the VirtualSize is more than SizeOfRawData, the PE loader will load all the SizeOfRawData (The total size of the section on the desk) and the rest of the VirtualSize will be Zeroes in memory, right ?
Ok, let's say that the VirtualSize is 1800h, and the SizeOfRawData is 1700h, the PE loader will load 1800h byte in memory, the total 1700h of SizeOfRawData , Plus 100h of zeroes, right ?

Here I got another question

as I understood, the PE loader will align the section size in memory according to the "SectionAlignment" in the PE header, so if the SectionAlignment is 1000h (as usual), The PE loader will load that section which has ( 1800h VirtualSize ) in memory (the last 100h bytes are "zeroes" ) , and Align it to the SectionAlignment (1000h), so it will be 2000h in memory, right ?
so there will be an extra 200 bytes again .
Does these 200h bytes will filled with "zeroes" too ?

Please anyone clearfy this for me

and Thank you again.
naides
May 10th, 2007, 07:12
Not only filled with zeros.
(It IS, at least initially filled with zeros, for alignment purposes)

But the Extra space that VirtualSize provides may, and often is, filled with data that is generated and initialized at run time, right after the file is loaded: It may be data read for another file, data uncompressed from the resources section, data calculated based on some algorithm present in the initialization code, you name it.

There is no reason to maintain all that blank slate, empty space of the un-initialized variables in the disk file, so it gets allocated only when the program loads in memory.

On the other hand sometimes the SizeofRawdata section in disk gets padded with zeros also (much less this time), to align the sections' size to a round multiple of the page size, in order to increase the efficiency of the loader.
evlncrn8
May 10th, 2007, 09:15
next question -> where did my .data? go ;p
TBone
May 10th, 2007, 10:40
I was about to impart a warning that you shouldn't count on the loader zero-filling the excess virtual address space, even if it appears that this is what it actually does. But then I ran a search for "zero-filled" on the PECOFF (v8.0) specification and came across these relevant bits:

Code:


Offset: 8     Size: 4     VirtualSize

The total size of the section when loaded into memory.

If this value is greater than SizeOfRawData, the section

is zero-padded. This field is valid only for executable

images and should be set to zero for object files.

...

Offset: 16    Size: 4     SizeOfRawData

The size of the section (for object files) or the size of the

initialized data on disk (for image files). For executable

images, this must be a multiple of FileAlignment from the

optional header. If this is less than VirtualSize, the re-

mainder of the section is zero-filled. Because the

SizeOfRawData field is rounded but the VirtualSize field is

not, it is possible for SizeOfRawData to be greater than

VirtualSize as well. When a section contains only un-

initialized data, this field should be zero.

So, apparently you *can* count it being zero-filled at the time the section is loaded. Of course, whether or not it's still zero-filled at the EP or OEP is another thing altogether

Everything you never wanted to know about PECOFF:
http://www.microsoft.com/whdc/system/platform/firmware/PECOFF.mspx

Quote:
[Originally Posted by 'evlncrn8']next question -> where did my .data? go ;p

It is very dark. If you continue your .data is likely to be eaten by a grue.
LLXX
May 11th, 2007, 01:42
Quote:
[Originally Posted by TBone;65578]So, apparently you *can* count it being zero-filled at the time the section is loaded. Of course, whether or not it's still zero-filled at the EP or OEP is another thing altogether
It should be, unless TLS callbacks are present.

P.S. I've noticed that the M$ linker is rather stupid. Declaring a char[4096] will bring 4096 null bytes into the PE, which could've been eliminated with the zero-fill feature. LordPE's Rebuild PE option can fix this.
disavowed
May 11th, 2007, 03:20
Quote:
[Originally Posted by LLXX;65593]I've noticed that the M$ linker is rather stupid. Declaring a char[4096] will bring 4096 null bytes into the PE, which could've been eliminated with the zero-fill feature.

No, because the zero-fill feature is undocumented and thus may change in future versions of Windows.
Mo7attem
May 11th, 2007, 08:29
Hi all, and thank you for your answers

naides I'm not sure whether I got an answer for my question or not

this was my question:
Quote:
if we have a section with VirtualSize 1800h, and SizeOfRawData 1700h, the PE loader will align the section size in memory according to the "SectionAlignment" in the PE header, so if the SectionAlignment is 1000h (as usual), The PE loader will load that section which has ( 1800h VirtualSize ) in memory (the last 100h bytes are "zeroes" ) , and Align it to the SectionAlignment (1000h), so it will be 2000h in memory, right ?
so there will be an extra 200 bytes again .
Do these 200h bytes will filled with "zeroes" too ?


naides you've said:
Quote:

Not only filled with zeros.
(It IS, at least initially filled with zeros, for alignment purposes)


what do you mean with this ? do you mean the Extra 100h byte of the VirtualSize, or the last 200h bytes which is for alignment purposes , or both of them ?

can you (or anyone) give an answer please ?
blabberer
May 11th, 2007, 09:55
the best way to find answer is to take a pe and hand modify all you want with a hexeditor crash the application a few times till you get it all right with the values you want to be there and then load it in a debugger and look

look for solar designers 132 bytes real pe file that can run in windows os and fetch a file from internet or look for ero carreras graphic analysis of the mentioned pe file you could get a rough idea of what windows loader looks for and how it behaves under extreme circumstances

especially you will learn how to hand modify a pe to be what you wish

if sizeofrawdata = 0x1700 file alignment = 0x200 the file in disk will have 100 bytes extra filled with 0000

so the size in disk of that specific section will be 0x1800

now when loaded loads it it will load it to file alignment which normally is 0x1000

so in virtual memory you will have 0x2000 bytes

so when you are viewing it in runtime you will have 0x900 bytes of zero padded space out of which you can use 0x100 bytes only to write back if you are intending to use this space as cave
TBone
May 11th, 2007, 13:04
The PE specification doesn't actually seem to address the issue of where the section ends. You are required to start on a SectionAlignment boundary, but your VirtualSize can be an arbitrary value. That said, what actually happens is that loader will extend the section to the next SectionAlignment boundary, with the leftover space being zero-filled.

The underlying reason for this is that (with the PSE and PAE flags turned OFF), the page size on x86-based systems is 4kB. This is equivalent to 0x00001000, the default value for SectionAlignment on x86 systems. At a minimum, the OS has to map virtual memory in 4kB chunks. So if you request a section of 0x0800 bytes (2kB decimal), the OS still has to map an entire 4kB page to create the section.

From your example, let's say that the section was loaded at address 0x00401000. A map of the section would look like this:
0x00401000..0x004026FF = raw data from image file
0x00402700..0x00402FFF = zeros

If this is the last section, a read or write at address 0x00403000 will fail with an access violation, because you tried to read beyond the end of the page to unmapped memory. In Ollydbg, the status line (just above the dump) for that instruction would look something like:
ds:[00403000] = ????

Last, but not least, you can specify a SectionAlignment smaller than the architecture's minimum page size, but there are extra requirements if you do. I really don't recommend this.
LLXX
May 12th, 2007, 02:23
Quote:
[Originally Posted by disavowed;65595]No, because the zero-fill feature is undocumented and thus may change in future versions of Windows.
Quote:
[Originally Posted by PE File Format Reference, Microsoft Corp.]VirtualSize
Total size of the section when loaded into memory. If this value is greater than Size of Raw Data, the section is zero-padded. This field is valid only for executable images and should be set to 0 for object files.

Quote:
[Originally Posted by PE File Format Reference, Microsoft Corp.]SizeOfRawData
Size of the section (object file) or size of the initialized data on disk (image files). For executable image, this must be a multiple of FileAlignment from the optional header. If this is less than VirtualSize the remainder of the section is zero filled. Because this field is rounded while the VirtualSize field is not it is possible for this to be greater than VirtualSize as well. When a section contains only uninitialized data, this field should be 0.
This is taken from v6 of the spec, I just checked the latest v8 and it's almost exactly the same with slight grammatical changes. Zero-fill is still mentioned though.
blabberer
May 12th, 2007, 11:46
well like i said it doesnt take a lot of effort to confirm this first hand

the minimum modification thats needed in a simple sample to check the header modification of virtual size greater than size of raw data is a 4 byte patch

the example binary thats taken here is iczelions tut -02 msgbox.exe

the modifications with annotation that makes the exe load in a debugger is
Code:


test:\>fc msgbox.exe msgboxmod1.exe

Comparing files msgbox.exe and MSGBOXMOD1.EXE

00000101: 40 50  <---- size of image 

000001B1: 00 10  <----- modifiec virtual size to 1026 instead of 26

000001DD: 20 30  <---- shifted further sections down by 0x1000

00000205: 30 40  <----- ditto



this file wont run will crash with access vioaltion 

but load happily in ollydbg 

ollydbg will warn like this

Log data, item 4

 Address=00401000

 Message=  Code size in header is 00000200, extending to size of section '.text'



the full disassembly of the said file will be 

00401000 >/$  6A 00         PUSH 0

00401002  |.  68 00304000   PUSH msgboxmo.00403000                   ;  ASCII "\ "

00401007  |.  68 19304000   PUSH msgboxmo.00403019

0040100C  |.  6A 00         PUSH 0

0040100E  |.  E8 0D000000   CALL msgboxmo.00401020

00401013  |.  6A 00         PUSH 0

00401015  |.  E8 00000000   CALL msgboxmo.0040101A

0040101A  \$  FF25 00204000 JMP DWORD PTR DS:[402000]

00401020   $  FF25 08204000 JMP DWORD PTR DS:[402008]



which still points to old values and the imports arent resolved because of conflicts in virtual addresses



DS:[00402000]=00000000

Local call from <ModuleEntryPoint>+15

msgboxmo.<ModuleEntryPoint>+1A



DS:[00402008]=00000000

Local call from <ModuleEntryPoint>+0E





test:\>


the next few modification will be change the import table addresses in pe header to make this work
Code:


test:\>fc msgbox.exe msgboxmod2.exe

Comparing files msgbox.exe and MSGBOXMOD2.EXE

00000101: 40 50

00000131: 20 30

00000189: 20 30

000001B1: 00 10

000001DD: 20 30

00000205: 30 40

0000041D: 20 30

00000423: 20 30



test:\>


and finally you get a working executable with these modifications

Code:


Comparing files msgbox.exe and MSGBOXMOD5.EXE

000000E1: 20 30  000000E0    00300000    DD 00003000          ;  BaseOfData = 3000 

00000101: 40 50  00000100    00500000    DD 00005000          ;  SizeOfImage = 5000 (20480.)

00000131: 20 30  00000130    10300000    DD 00003010          ;  Import Table address = 3010

00000189: 20 30  00000188    00300000    DD 00003000          ;  Import Address Table address = 3000

000001B1: 00 10  000001B0    26100000    DD 00001026          ;  VirtualSize = 1026 (4134.)

000001DD: 20 30  000001DC    00300000    DD 00003000          ;  VirtualAddress = 3000

00000205: 30 40  00000204    00400000    DD 00004000          ;  VirtualAddress = 4000

00000404: 30 40  00000402    68 00404000     PUSH 404000

00000409: 30 40  00000407    68 19404000     PUSH 404019

0000041D: 20 30  0000041A    FF25 00304000   JMP DWORD PTR DS:[403000]

00000423: 20 30  00000420    FF25 08304000   JMP DWORD PTR DS:[403008]

00000601: 20 30

00000609: 20 30

00000611: 20 30

0000061D: 20 30

00000621: 20 30

00000625: 20 30

00000631: 20 30

00000635: 20 30

0000064D: 20 30

00000655: 20 30



00000600  5C 30 00 00 00 00 00 00 78 30 00 00 00 00 00 00  \0......x0......

00000610  4C 30 00 00 00 00 00 00 00 00 00 00 6A 30 00 00  L0..........j0..

00000620  00 30 00 00 54 30 00 00 00 00 00 00 00 00 00 00  .0..T0..........

00000630  86 30 00 00 08 30 00 00 00 00 00 00 00 00 00 00  †0..0..........

00000640  00 00 00 00 00 00 00 00 00 00 00 00 5C 30 00 00  ............\0..

00000650  00 00 00 00 78 30 00 00 00 00 00 00              ....x0......





you would still see some warnings like size of code in header is 200 extending

size to size of .text section 

etc in ollydbg 



if you modify sizeofcode then probably you have to manually fill the diskcopy with 

padding zeroes if i remember correctly 



thats how you extend and make caves in executable



these methods wont work if the exe has relocation tables

as it is you have to manually modify the fixup datas too 

in .reloc section so that loader knows how to fixup correctly


this will have 0x2000 bytes in virtual memory ZERO PADDED

Code:


00402FFC    0000            ADD BYTE PTR DS:[EAX],AL

00402FFE    0000            ADD BYTE PTR DS:[EAX],AL


and with these modification it should work too by double clicking the exe
disavowed
May 14th, 2007, 09:35
Quote:
[Originally Posted by LLXX;65625]This is taken from v6 of the spec, I just checked the latest v8 and it's almost exactly the same with slight grammatical changes. Zero-fill is still mentioned though.

okay, i'm wrong
BlackRoc
June 2nd, 2007, 08:00
This thread seems close to my question, so here goes.
For a long time, I have used the space between SizeOfRawData = size on disk and VirtualSize = size in memory to store code in a cave that I want to add to an exe. I typically start up LordPE, increase VSize of .text, save and begin writing my code..

The other day I did exactly that with some code that checked itself first via a separate dll and again with the main exe. I opened caves in both with LordPE. Everything worked fine, but I'd hacked up the exe changes I made in the main and wanted to start fresh so I did it again and suddenly it wouldn't run. To make a long story short, it turned out my first attempt at setting VirtualSize on main.exe with LordPE had apparently been screwed up and I didn't save it. I'd installed my code in the cave, I'd debugged and modified it with Olly, I'd run it (and can still run it) all with a VSize in the header too small to include my code in the cave, yet the code is there and is functional.

Apparently some part of the dll or main.exe is checking the header and it would crash if I modified the header. I inadvertently screwed up the LordPE header change and got past that check. ( I assume I screwed up, I can't see how it could have changed back) I can track that down, but I don't understand what I'm seeing. I pulled up some more tuts to verify what I thought was *supposed* to happen - namely that only the VSize would be loaded into memory. That's what they all seem to say, but my cave code is loading and running just fine, even though it's beyond the VSize.

Can anyone explain this? Was I wasting my time changing the header with LordPE each time? Does it load the entire SizeOfRawData = RSize on disk into memory regardless of VirtualSize = VSize in memory?

Thanks for helping me understand this.
disavowed
June 2nd, 2007, 12:24
It will be rounded up for page alignment.
BlackRoc
June 2nd, 2007, 14:59
Quote:
[Originally Posted by disavowed;66136]It will be rounded up for page alignment.


Thanks. But isn't the reason there's a difference between VSize and RSize because of page alignment? That would mean I don't need to bother to increase the size of VSize to match-1 the RSize. All the tutorials on using a cave for extra code seem to start off with that initial step to get space.

Here's a typical description:

Quote:
Looking at the main section table again, we see that the next PE section, .rdata starts at raw file offset 6000h, this is right after the .text section ends.

The intresting thing to note, is that the virtual size (the size in memory) of the .text section is not quite using up the entire 5000h bytes that are laid out for it in the file. This means that there is a slight gap between the two sections in the file.
...
This extra space is totally unused, and not loaded into memory. Next we need to assure that whatever instructions we place in there, will be loaded into memory. We do this by altering the .text section attributes in the PE header with LordPE.

Right now the virtual size of this section is only 4E5E, because that is all the compiler needed. We need a little more, so lets change the virtual size of the .text section all the way up to 4FFF which is the max size we can use seeing how the entire raw size is only 5000.

Once that is done, we now have a suitable place to store our patch code.


Is this wrong, or am I missing something?
blabberer
June 6th, 2007, 12:10
as far as i know there is no need for changing v size to match to rsize -1

try the following and you will see what im saying is true

open win.exe (iczelion tutorial-3) in ollydbg

right click view executable
Code:


see the .data section

00000208    2E 64 61 74>ASCII ".data"        ; SECTION

00000210    28000000    DD 00000028          ;  VirtualSize = 28 (40.)

00000214    00300000    DD 00003000          ;  VirtualAddress = 3000

00000218    00020000    DD 00000200          ;  SizeOfRawData = 200 (512.)

0000021C    00080000    DD 00000800          ;  PointerToRawData = 800

00000220    00000000    DD 00000000          ;  PointerToRelocations = 0

00000224    00000000    DD 00000000          ;  PointerToLineNumbers = 0

00000228    0000        DW 0000              ;  NumberOfRelocations = 0

0000022A    0000        DW 0000              ;  NumberOfLineNumbers = 0

0000022C    400000C0    DD C0000040          ;  Characteristics = INITIALIZED_DATA|READ|WRITE


vsize is definately small than rsize

lets add some crap

ctrl+g type in an arbitrary number lets say 8a0 (note pointer to rawdata is at 800)

and fill the place with some test string

Code:


000008A0  62 6C 61 63 6B 72 6F 63 27 63 65 64 62 6C 61 63  blackroc'cedblac

000008B0  6B 72 6F 63 27 63 65 64 62 6C 61 63 6B 72 6F 63  kroc'cedblackroc

000008C0  27 63 65 64 62 6C 61 63 6B 72 6F 63 27 63 65 64  'cedblackroc'ced

000008D0  62 6C 61 63 6B 72 6F 63 27 63 65 64 62 6C 61 63  blackroc'cedblac

000008E0  6B 72 6F 63 27 63 65 64 62 6C 61 63 6B 72 6F 63  kroc'cedblackroc

000008F0  27 63 65 64 62 6C 61 63 6B 72 6F 63 27 63 65 64  'cedblackroc'ced

00000900  62 6C 61 63 6B 72 6F 63 27 63 65 64 62 6C 61 63  blackroc'cedblac

00000910  6B 72 6F 63 27 63 65 64 62 6C 61 63 6B 72 6F 63  kroc'cedblackroc

00000920  27 63 65 64 62 6C 61 63 6B 72 6F 63 27 63 65 64  'cedblackroc'ced

00000930  62 6C 61 63 6B 72 6F 63 27 63 65 64 62 6C 61 63  blackroc'cedblac

00000940  6B 72 6F 63 27 63 65 64 62 6C 61 63 6B 72 6F 63  kroc'cedblackroc

00000950  27 63 65 64 62 6C 61 63 6B 72 6F 63 27 63 65 64  'cedblackroc'ced

00000960  62 6C 61 63 6B 72 6F 63 27 63 65 64 62 6C 61 63  blackroc'cedblac

00000970  6B 72 6F 63 27 63 65 64 62 6C 61 63 6B 72 6F 63  kroc'cedblackroc

00000980  27 63 65 64 62 6C 61 63 6B 72 6F 63 27 63 65 64  'cedblackroc'ced

00000990  62 6C 61 63 6B 72 6F 63 27 63 65 64 62 6C 61 63  blackroc'cedblac


right click save file -> winmod.exe

then open this new file you will see your nick still intact
we havent touched peheader at all
BlackRoc
June 6th, 2007, 12:37
Quote:
[Originally Posted by blabberer;66208]as far as i know there is no need for changing v size to match to rsize -1

Yes, this matches my own testing. You will see from the tut I quoted that it is often said that this cave area will not be loaded into memory, but perhaps it changes from W98-WME-W2K-WXP or perhaps it has always been just a myth. I have since tried testing it out different ways, and it seems to always load on WXP, no matter how much of the cave I tried to use. I also found some tuts that use this cave area without adjusting the PE header, so not all have "fixed" the PE header. I'd just like to be sure that some day it doesn't decide to skip loading my code because the PE header isn't correct.