Dissecting the PE File Format - 12

PACKERS

In this post I will cover the effects of a simple packer and 2 ways of patching a packed executable - either by unpacking first or in-line patching. I use UPX1.25 since this is an executable compressor and doesn't use advanced protection mechanisms

First, we scan our app with PEiD

Next, we pack our app with UPX. This is a command line utility so we open a DOS box where our app is an type upx <filename.exe>

We notice the file down from 225kb to 91kb and in PEID we see this:

PEBrowse Pro shows that there are only 3 sectons UPX0, UPX1 and .rsrc.  The resource section now contains the import directory but for each DLL there are only two imported functions - the others have disappeared.

Note the .rsrc section has retained the original name even though the others have changed. Interestingly this dates back to a bug in the LoadTypeLibEx function in oleaut32.dll in win95 in which the string ".rsrc" was used to find and load the resource section. This created an error if the section was renamed. Although this bug has been fixed it seems most packers do not rename the rsrc seciton for compatibility reasons.

By opening the app in LordPE editor and pressing the compare button we can open an original copy of our app and see the changes made to the headers.

Open the app in Olly. Click OK on the warning which tells you that the executable is probably packed and we land at the entry point.

UPX has compressed our app and appended the code with a stub containing the decompression algorithm. The entry point of this app has been changed to the start of the stub and after the stub has done its job, execution jumps to the original entry point to start our now unpacked program.

The rationale for dealing with this is to let the stub decompress the app in memory and then dump the memory region to a file to get the unpacked copy of the app. However, the app will not run straight away because the dumped file will have its sections aligned to memory page boundaries rather than file alignment values, the entry point still points to the decompression stub and the Import Directory is also wrong.

Note at our entry point in Olly we have Pushad. This stands for push all double and instructs the CPU to store the contents of all the 32 bit registers on the stack staring with EAX and ending with EDI. Following this, the stub does its job and then ends with a POPAD instruction before jumping to the OEP. POPAD copies the contents of the register back from the stack. This means that the stub will have restored everything back the way it was and exited without trace before running the app. Since this method is ideal in this situation, it is common to other simple packers.

From the time of the first PUSHAD instruction, the contents of the stack at that level must remain untouched until accessed by the final POPAD. If we put a hardware breakpoint on the first 4 bytes of the stack at the time of the PUSHAD Olly will break when the same 4 bytes are accessed by the POPAD instruction and we will be right in front of the jump to OEP.

First, press F7 to execute the PUSHAD instruction. Next we place the breakpoint. The ESP register always contains the top of the stack so, rightclick on ESP and select follow in dump - this puts the stack in hexdump window.

Now highlight the DWORD of the stack, rightclick and select breakpoint, hardware on access, DWORD:

Next, run the app by pressing F9 and Olly breaks directly before the jmp to OEP. The OEP shown here ahs ImageBase 400000h added to it so to make it an RVA we subtract it which leaves us with 2ADB4h

If you want to cheat, for some packers simply scroll to the end of the code in the CPU window in Olly and just before all the zero padding starts you will see the POPAD instruction.

Next, we single step once so we are the OEP and dump the app using Ollydump plugin. Just click on plugins, Ollydump and select dump debugged process. In the next box we will deselect fix raw size and rebuild imports to illustrate some points of interest:

Note that OllyDump has already worked out the base address an size of image and has offered to correct the entry point for us. Press dump and save the file.

Unfortunately we see that something is wrong because our file has lost its icon and if we try to run it we get an error.

This is because of the alignment issues mentioned earlier - the filesize has also increased as a result. Open the app in LordPE and look at the sections. The raw offset and the raw size values are wrong. We will have to make the Raw values equal to the virtual values for each section for the app to work. Rightclick the UPX0 section and select edit header.

Now make RawOffset equal VirtualAddress and RawSize equal VirtualSize. Repeat for the other sections, save and exit (this is what fix raw sizes checkbox in Olly means). now the icon has returned and we get a different error when we try to run it. "The application failed to initialize properly." This is because the imports still need rebuilding.

To do this, we use ImpRec. It needs to attach to a running process and also needs the packed file to find imports. Start ImpRec and follow these steps:

1. select basecalc.exe in the box at the top (it should still be running in Olly.) 

2. Next enter our OEP (2ADB4) in the appropriate box 

3. Press the "IAT AutoSearch" button and click OK on the messagebox 

4. Press the "Get Imports" button 

5. Press "Show Invalid" - in this case there are none 

6. Press "Fix Dump" and select basecalc_dmp.exe in the open dialogbox 

7. Exit.

ImpRec will save a fixed copy of our dumped file appended with "_". So run and test it. If we examine the file, we see an extra section named "mackt" - this is where ImpRec puts the new import data. 

Since UPX is purely a compressor, it has taken the existing import data and stored it in the resource section without encrypting or damaging it. This is why ImpRec finds all valid imports without resorting to tracing or rebuilding - it has taken the import directory from the packed executable in memory and transferred it to the new section in the unpacked executable.

PEiD now shows Borland Delphi instead of UPX.

These are the steps necessary to unpack an executable packed with a simple compressor. More advanced packers add various protection schemes to this e.g. antidebugging and anti-tampering tricks, encryption of code and IAT, stolen bytes, API redirection etc. which are beyond the scope of this post.

If it is necessary to patch a packed executable, it may be possible to avoid unpacking it first by using a technique called "inline-patching". This involves patching the code at runtime in memory after the decompression stub has done its work and then finally jumping to the OEP to run the app. In other words we wait until the app is unpacked in memory, jump to patching code we have injected, then finally jump back to the OEP.

To illustrate this we will inject code in the packed executable to pop up a messagebox and let us know when the app is unpacked in memory. Clicking OK will then jump to the OEP and the app will run normally.

The first task is to find some free space for our code so open the packed app in the hex editor and look for a suitable "cave". Free space at the end of a section is better as it is less likely to be used by the packer and is extensible by enlarging the section if necessary. You can see how efficient UPX is - there is hardly any free space - but a small cave exists here. Now add the text "Unpacked" and "Now back to OEP" in the ASCII column of the hex editor as shown:

This will mark our spot for the patch in Olly without having to worry about calculating RVAs. Save changes and open the app in Olly. Rightclick in the hex window and select search for binary string. Now enter "Unpacked" and note the VA of the two strings. In the CPU window, right click and select go to expression. Enter the address of the first string and you will see the two strings in hexadecimal form. Olly has not analyzed this properly so it displays nonsense code next to it. Highlight the next free row underneath and press spacebar to assemble the following instructions.

PUSH 0

PUSH 440C30 [address of first string]

PUSH 440C40 [address of second string]

PUSH 0

CALL MessageBoxA

JMP 42ADB4

Make a note of the address of our first PUSH instruction - 440C4E. Our code should look like this

Now rightclick and select copy to executable, selection. In the new window rightclick and select save file etc. If we check in the hexeditor we see that our code has been added.

Finally we need to change the JMP at the end of theUPX stub to go to our code. Find it as shown earlier, double click the JMP instruction to assemble and change the address to 440C4E. Save changes again and run app to test it.

This ends the entire PE File Format Series :-)

Dissecting the PE File Format - 11

Adding Import To an Executable

This is most often used in the context of patching a target app where we dont have the APIs we need. To recap, the minimum information needed by the loader to produce a valid IAT is:

1. Each DLL must be declared with an IMAGE_IMPORT_DESCRIPTOR(IID), remembering to close the Import Directory with a null-filled one.

2. Each IID nneds at least Name1 and FirstThunk fields, the rest can be set to 0 (setting OriginalFirstThunk = FIrstThunk i.e. duplicating the RVAs also works)

3. Each entry of the FIrstThunk must be an RVA to an Image_Thunk_Data (the IAT) which in turn further contains an RVA to the API name. The name will be a null-terminated ASCII string of variable length and preceded by 2 bytes (hint) which can be set to 0.

4. if IIDs have been added then the isize field of the import table in the Data Directory may need changing. The IAT entries in the Data Directory need not be altered.

Writing new import data in a hex editor and then pasting into your target can be very time-consuming. There are tools which can automate this (SnippetCreator, IIDKing etc) but to understand the concept its best to do it manually. The main task is to append a new IID onto the end of the import table - you need 20 bytes for each DLL used, not forgetting 20 bytes for the null-terminator. In nearly all cases there will be no space at the end of the existing import table so we will make a copy and relocate it somewhere there is space.

Step 1 - Create space for a new IID

This involves the following steps:

1. Move all the IIDs to a location where there is plenty of space. This can be anywhere; at the end of the current .idata section or an entirely new section. 
2. Update the RVA of the new Import Directory in the Data Directory of the PE header.
3. If necessary, round up the size of the section where you've put the new Import table so everything is mapped in memory (e.g. VirtualSize of the .idata section rounded up 1000h)
4. Run it and if it works we move on to step 2. if it doesn't check the injected descriptors are mapped in memory and that the RVA of the import directory is correct.

NOTE - The IIDs, FirstThunk and OriginalFirstThunk contain RVAs - RELATIVE ADDRESSES - which means you can cut and paste the Import Directory (IIDs) wherever you want in the PE file (taking into account the destination has to be mapped into memory) and simply changing the RVA (and size if necessary) of the Import Directory in the Data Directory will make the app run perfectly.

Back to the hex editor, the first IID and the null terminator are outlined in red and you see that there is no space after the null IID.

However, there is a large amount of space at the end of the .idata section before .rdata starts. We will copy and paste the existing IIDs shown above to offset 2C500h at this new location:

 

To convert the new offset to an RVA

VA = Raw Offset - RawOffsetOfSection + VirtualOffsetOfSection

 = 2C500 - 2AC00 + 2D000 = 2E900h

So change the virtual address of the import table in the data directory from 2D000 to 2E900. Now edit the .idata section header and make virtual size equal to RawSize so the loader will map the whole section in. Run the app to test it.

Step 2 - Add the new DLL and function details

This involves the following steps:

1. Add null-terminated ASCII string of the names of your DLL and function to a free space in the .idata section. The function name will actually be an Image_Import_By_Name structure preceded by a null word (the hint field)
2. Calculate the RVAs of the above strings
3. Add the RVA of the DLL name to the Name1 field of your new IID
4. Find another DWORD sized space and put in it the RVA of the hint/function name. This becomes the Image_Thunk_Data or IAT of our new DLL.
5. Calculate the RVA of the above Image_Thunk_Data DWORD and add it tot he FirstThunk field of your new IID.
6. Run the app to test and your new API is ready to be called.

In order to fill our new IID we need at the very least Name1 and FirstThunk fields (the others can be nulled). The Name1 field contains the RVA of the name of the DLL in null-terminated ASCII. The FirstThunk field contains the RVA of an Image-Thunk Data Structure which in turn contains yet another RVA of the name of the function in null-terminated ASCII. The name is preceded by 2 bytes (Hint) which can be set to zero.

For e.g. we want to use the function LZCopy which copies a source file to a destination file. if the source file is compressed with the Microsoft File Compression Utility, this function creates a decompressed destination file. If the source file is not compressed, this function duplicates the original file.

This function resides in lz32.dll which is not currently used by our app. Therefore we first need to add strings for the names "Lz32.dll" and "LZCopy". We scroll upwards in the hex editor from the new import table towards the end of the preexisting data and add the DLL name then the function name onto the end. Note the null bytes after each string and the null WORD before the function name.

Now we need to calculate the RVAs of these:

RVA = RawOffset - RawOffsetOfSection + VirtualOffsetOfSection + ImageBase

RVA of DLL Name - 2c420 - 2ac00 + 2d000 = 2E820 (20 E8  02 00 in reverse)

RVA of Function name - 2c430 - 2ac00 + 2d000 = 2E830 (30 E8 02 00 in reverse)

The first one can go into the name1 field of our new IID but the second must go into an Image_Thunk_Data structure, the RVA of which we can put into the FirstThunk field (and OriginalFirstThunk) of our new IID. We will put the Image_Thunk_Data structure below the function name string at the offset 2C440 and calculate the RVA which we will put in the FirstThunk

RVA of Image_Thunk_Data = 2C440 - 2AC00 + 2D00 = 2E840 (40 E8 02 00 in reverse)

If we fill the data in the hexeditor:

Finally save changes, run the app to test and re-examine the imported functions in PEBrowse

In order to call the new function, we can use the following code:

CALL DWORD PTR [XXXXXXX] where XXXXXXXX = RVA of Image_Thunk_Data + ImageBase

Final Note: Even if we had added a function used by a DLL which was already in use e.g. kernel32.dll we would still need to create a new IID for it to enable us to create a new IAT at a convenient location as above.

In the next pose we talk about packers.

Dissecting the PE File Format - 10

Adding Code to a PE File

It might be necessary to add code to a program  to either crack a protection scheme or to add functionality.  There are 3 main ways to add code to an executable:

1. Add to an existing section when there is enough space for your code

2. Enlarge an existing section when there is not enough space

3. Add an entirely new section

Add to an existing section

We need a section in the file that is mapped with execution privileges in memory so simplest is to try the CODE section. We then need an area in the section occupied by 00 byte padding. This is the concept of "caves". To find a suitable cave, look at the CODE section details in LORDPE:

Here we see that the VirtualSize is slightly less than SizeOfRawData. The virtual size represents the amount of actual code. The size of the raw data defines the amount of space taken up in the file sitting on your hard disk. Note that the virtual size in this case is lower than that on the hard disk. This is because compilers often have to round up the size to align a section on some boundary. In the hexeditor at the end of the code section we see:

This extra space is totally unused and not loaded into memory. We need to ensure that the instructions we place there will be loaded into memory. We do this by altering the size attributes. Right now the virtual size of this section is only 29E88, because that is all the compiler needed. We need a little more, so in LordPE change the virtual size of the code section all the way up to 29FFF which is the max size we can use (the entire raw size is only 2A000). To do this rightclick  the CODE line and select edit header, make the changes click save and enter.

Once that is done, we have suitable place for our patch code. The only thing we have changed is the VirtualSize DWORD for the CODE section in the Section Table. This could have also been done manually with a hex editor.

Next, we add a small ASM stub that hijacks the entry point and then just returns execution to the original Entry Point. We will do this in Olly.

First we note that the entry point is 0002ADB4 and ImageBase is 400000. When we load the app in Olly the EP will therefore be 0042ADB4. We will add the following lines and then change the entry point to the first line of code:

MOV EAX,0042ADB4 ; Load in EAX the Original Entry Point (OEP) 

JMP EAX ; Jump to OEP 

We will put them at 0002A300h as seen in the hexeditor above. To convert this raw offset to an RVA for use in Olly use the following formula:

RVA = raw offset - raw offset of section + virtual offset of section + ImageBase

 = 2A300h - 400h + 1000h + 400000h = 42AF00h

So load the app in Olly and jump to our target section (press Ctrl + G and enter 42aF00). Press space and type in the first line of assembly and click assemble. The next line should now be highlighted and do the same here.

Now rightclick, select copy to executable and all modifications. Click copy all then a new window will open. Rightclick in the new window and select save file. Now back in LordPE or a hex editor, change the entry point to 2AF00. Now run the app to test it and reopen in Olly to see your new entry point.

Enlarging an Existing Section:

If there is not sufficient space in the text section, you have to extend it. This poses a number of problems.

1. If the section is followed by other sections then you will need to move the following sections to make room.

2. There are various references within the file headers that will need to be adjusted if you change the file size. 

3. References between various sections (such as references to data values from the code section) will all need to be adjusted. This is impossible to do without recompiling and relinking the original file. 

Most of these problems can be avoided by appending to the last section of the exe file. It is not relevant what that section is as we can make it suit our needs by changing the Characteristics field in the Section Table either manually or with LordPE.

First we locate the final section and make it readable and executable. As we said earlier, the code section is ideal for a patch because its characteristics flags are 60000020 which means code, executable and readable. However, if we are to put code and data into this section we could get a page fault since it is not writeable. To alter this we would need to add the flag 80000000 which gives a new value E0000020 for code, executable, readable and writeable.

Likewise if the final section is .reloc then the flags will be 42000040 for initialized data, discardable and read-only. In order to use this section we must add code, executable and writeable and we must subtract discardable to ensure that the loader maps this section to memory. This gives us a new value of E0000060.

This can either be done manually by adding up flags and editing the Characteristics field of the Section header with the hexeditor or LordPE will do it. In our example, the last section is resources.

This gives us a final characteristic value of F0000060. Above we see the raw size (on disk) of this section is 8E00 bytes but all of this seems to be in use (the VirtualSize os the same). Now edit these and ad  100h bytes to both to extend the section. The new value is 8F00h. There are some other important values which need to be changed. The SizeOfImage field in the PE header needs to be increased by the same amount from 3CE00 to 3CF00.

There are 2 other fields not shown in LordPE which are less critical; SizeOfCode and SizeOfInitializedData in the OptionalHeader. The app will still run without these being altered but you may wish to change them for completeness. We will have to do it manually. Both are DWORDs at offset 1C and 20 from the start of the PE header.

The values are 0002A000 and 0000DE00 respectively. Add 100h on to these. With reverse bytes the values are: 00 A1 02 00 and 00 00 DF 00. Finally copy and paste 100h of 00 bytes onto the end of the section and save changes. Run the file to test for errors.

Adding a new section

In some circumstances you may need to make a copy of an exiting section to defeat self-checking procedures (such as SafeDisk) or make a new section to hold code when proprietary information has been appended to the end of the file.

The first job is to find the NumberOfSections field in the PE header and increase it by 1. Again, most of these changes can be done by LordPE or manually with an hexeditor. Now, in the hexeditor paste 100h of 00 bytes onto the end of the file and make a note of the offset of the first new line. In our case it is 00038200h. This will be the start of our new section and will go in the RawOffset field of the section header. While we are here, it is probably a good time to increase SizeOfImage by 100h bytes.

Next we need to find the section headers beginning at the offset F8 from the PE header. It is not necessary for these to be terminated by a header fill of zeros. The number of headers is given by NumberOfSections and there is usually some space at the end before the sections themselves start (aligned to the FileAlignment value). Find the last section and add a new one after it.

The next thing we need to do is decide which Virtual offset / Virtual Size / Raw Offset and Raw Size our section should have. To decide this, we need the following values:

Virtual offset of formerly last section (.rsrc): 34000h 

Virtual size of formerly last section (.rsrc): 8E00h 

Raw offset of formerly last section (.rsrc): 2F400h 

Raw size of formerly last section (.rsrc): 8E00h 

Section Alignment: 1000h 

File Alignment: 200h 

The RVA and the raw offset of our new section must be aligned to the above boundaries. The Raw Offset of the section is 38200h as we said above (which luckily fits with FileAlignment). To get the VirtualOffset of our section we have to calculate this: VirtualAddress of .rsrc + VirtualSize of .rsrc = 3CE00h. Since our SectionAlignment is 1000h we must round up to the nearest 1000 which makes 3D000h. So lets fill the header of our section.

The first 8 bytes will be Name1 (max. 8 chars e.g. "NEW" will be 4E 45 57 00 00 00 00 00 (byte order not reversed) 
The next DWORD is VirtualSize = 100h (with reverse byte order = 00 01 00 00) 
The next DWORD is VirtualAddress = 3D000h (with reverse byte order = 00 D0 03 00) 
The next DWORD is SizeOfRawData = 100h (with reverse byte order = 00 01 00 00) 
The next DWORD is PointerToRawData = 38200h (with reverse byte order = 00 82 03 00) 
The next 12 bytes can be left null 
The final DWORD is Characteristics = E0000060 (for code, executable, read and write as discussed above) 

Save changes, run to test for errors and examine in LordPE

Dissecting the PE File Format - 9

Navigating Imports On Disk

 In the hex editor, we navigate the import table. The RVA of the import directory is stored in the DWORD 80h bytes from the PE header which in our example is offset 180h and the RVA is 2D000H (as in the Data Directory post). We now have to convert that RVA to a raw offset to peruse the correct area of our file on disk. Check the Section Table to see which section the address of the import directory lies in. In our case, the Import Directory starts at the beginning of the .idata section and we know that the section table holds the raw offset in the PointerToRawData field. In our example, the offset is 2AC00h

The difference between the RVA and the Raw offset is 2D000h - 2AC00h = 2400h. Make a note for this as it will be used for converting further offsets.

At offset 2AC00 we have the import directory - an array of IMAGE_IMPORT_DESCRIPTORS each of 20 bytes and repeating for each import library (DLL) until terminated by 20 bytes of zeroes. 

Each group of 5 dwords represent 1 IMAGE_IMPORT_DESCRIPTOR. The first shows that in this PE file OriginalFIrstThunk, TimeDateStamp and ForwarderChain are set to 0. Eventually we come to a set of 5 DWORDS all set to 0 (highlighted in red) signifying the end of the array. We see that we are importing from 8 DLLs. 

NOTE - The OriginalFirstThunk fields in our example are all set to zero. This is common for executables made with the Borland compiler and linker and is noteworthy for the following reason. In a packed executable the FirstThunk pointers will have been destroyed but can sometimes be rebuilt by copying the duplicate OriginalFirstThunks, There is a utility called First_Thunk Rebuilder which will do this. However, with Borland created files this is not possible because the OriginalFirstThunks are all zeros and there is no INT.

Back to our example above, the Name1 field of the first IMAGE_IMPORT_DESCRIPTOR contains the RVA 00 02  DF 30h (NB Reverse Byte Order). Convert this to a raw offset by subtracting 2400h (remember above) and we have 2b130h. If we look at our PE file we see the name of out DLL.

To continue, the FirstThunk field contains the RVA 00 02 D0 B4h which converts to Raw Offset 2ACB4h. Remember, this si the offset to array of DWORD-size IMAGE_THUNK_DATA structures - the IAT. This will either have its most significant bit set and the lower part will contain the ordinal number of the imported function, or if the MSB is not set it will contain yet another RVA to the name of the function (IMAGE_IMPORT_BY_NAME)

In our file, the DWORD at 2ACB4h is 00 02 D5 3E

This is another RVA which converts to Raw Offset 2B13E. This time it should be a null-terminated ASCII string. In our file, we see

So, the name of the first API imported from kernel32.dll is DeleteCriticalSection. You may notice the two  zero bytes before the function name. This is the hint element which is often set to  00 00.

All of this can be verified using PEBrowse Pro to parse the IAT as shown:

If the file has been loaded into memory, dumped and examined with the hex editor then the DWORD at RVA 2D0B4h which contained 3E D5 02 00 on disk would have been overwritten by the loader with the address of DeleteCriticalSection in kernel32.dll

Allowing for reverse byte order this is 7C91188A

NOTE:  functions in system DLLs always tend to start at addresses 7xxxxxx and stay the same each time the program is loaded.  However, they tend to change if you reinstall your OS and differ from one computer to another.

The addresses also differ according to OS, for example: 

OS                           Base of kernel32.dll 

Win XP SP1                  77E60000H 

Win XP SP2                  7C000000H 

Win 2000 SP4                79430000H 

Windows updates also sometimes change the base location of system DLLs. 

Navigating Imports in Memory

Load the example in Olly and look at the memory map

Note the address of the idata section is 42D000 which corresponds to the RVA 2D000 shown at the top of this post as VOffset.  This size has been rounded up to 200 to fit memory page boundaries.

The main (CPU) window of Olly will only show the IAT if it lies in the executable CODE section, however in most cases it will be in its own section e,g, idata. You can view the IAT in Olly's hex-dump by right clicking the appropriate section in memory map and selecting Dump in CPU. Now rightclick in the hex window and select Long>Address and you will see the IAT in a readable list. 

This makes finding the beginning and the end of the IAT easy and is useful when using ImpREC as the IAT AutoSearch function can be inaccurate. It is good to be able to check the beginning and endpoint to avoid having to type in a large size value which will give many false negatives.

The names window (ctrl + N) will show you imported functions:

Rightclicking any of these and selecting find references to Import will show you the jump thunk stub and the instances in the code where the function is called (only 1 in this case)

Note: In the comment column you will see that Olly has determined that the kernel32.dll function DeleteCriticalSection is actually forwarded to RtlDeleteCriticalSection in ntdll.dll 

Rightclicking and selecting Follow Import in Disassembler will show you the address in the appropriate DLL where the function's code starts e.g. starts at 7C91188A in ntdll.dll

If we look at the call to DeleteCriticalSection at 00401B12 we see:

This is really "CALL 00401314" but Olly has already substituted the function name for us. 401314 is the address of the jump stub pointing to the IAT. Note it is a part of the jump thunk table described previously:

This is really "JMP DWORD PTR DS:[0042D0B4]" but again Olly has substituted the symbolic name for us. Address 0042D0B4 contains the Image_Thunk_Data Structure in the IAT which has been overwritten by the loader with the actual address of the function in kernel32.dll: 7C91188A. This is what we found earlier by rightclicking and selecting follow Import in Disassembler and also from the dumped file above.

In the next post we talk about Adding Code to a PE File

Dissecting the PE File Format - 8

The Loader


This post is not generally essential and only for those who want to dig deeper.

A brief overview of the stages involved in the loading process:

1. Read in the first page of the file with the DOS header, PE header, and section headers.
2. Determine whether the target area of address space is available, if not allocate another area.
3. Using info in the section headers, map sections of the file to appropriate places in the allocated address space.
4. If the file is not loaded at its target address (ImageBase), apply relocation fix-ups.
5. Go through the list of DLLs in the import sections and load any that aren't already loaded (recursive).
6. Resolve all imported symbols in the imports section
7. Create the initial stack and heap using values from the PE header.
8. Create the initial thread and start the process

What the loader does

When the executable is run, the windows loader creates a virtual address space for the process and maps the executable module from the disk into process' address space. It tries to load the image at the preferred base address but relocates it if that address is already occupied. The loader goes through the section table and maps each section at the address calculated by adding the RVA of the section to the base address. The page attributes are set according to the section's characteristic requirements. After mapping the sections in memory, the loader performs base relocations if the load address is not equal to the preferred base address in ImageBase.

The import table is then checked and any required DLLs are mapped into the process address space. After all the DLL modules are located and mapped in, the loader examines each DLLs export section and the IAT is fixed to point to the actual imported function address. If the symbol does not exist (rare), the loader displays an error. Once all the required modules have been loaded execution passes to the apps entry point.

The area of particular interest is that of loading the DLLs and resolving imports. This process is complicated and accomplished by various internal (forwarded) functions and routines residing in ntdll.dll which are not documented by Microsoft. Function forwarding is a way for M$ to expose a common Win32 API set and hide low level functions which may differ in different versions of the OS. Many familiar kernel32 functions such as GetProcAddress are simply thin wrappers around ntdll.dll exports such as LdrGetProcAddress which do the real work.

In order to see this in action you will need to install windbg and the windows symbol package or another kernel-mode debugger like SoftIce. You can only view these functions in Olly if you configure Olly to use the M$ symbolserver, otherwise all you see is pointers and memory addresses without function names. However, Olly is a user-mode debugger and will onyl show you whats hapenning when you app has been loaded and will not allow you to see the loading process itself. Although the functionality of Windbg is poor when compared to Olly, it does integrate well with the OS to show you the loading process.

The various APIs associated with loading an executable all converge on the kernel32.dll function LoadLibraryExW  which in turn leads to the internal function LdrpLoadDll in ntdll.dll. This function directly calls 6 subroutines LdrpCheckForLoadedDll, LdrpMapDll, LdrpWalkImportDescriptor, LdrpUpdateLoadCount, LdrpInitializeRoutines and LdrpClearLoadInProgress which perform the following tasks:

1. Check to see if the module is loaded

2. Map the module and supporting information into memory

3. Walk the module's import descriptor table (find other modules this one is importing)

4. Update the modules load count as well as any other brought in by this DLL.

5. Initialize the module.

6. Clear some sort of flag, indicating that the load has finished.

A DLL may import other modules that start a cascade of additional library loads. The loader will need to loop through each module, checking to see if it needs to be loaded and then checking its dependencies. This is where LdrpWalkImportDescriptor comes in. It has two subroutines - LdrpLoadImportModule and LdrpSnapIAT. First it starts with 2 calls to RtlImageDirectoryEntryToData to locate the bounds import descriptor and the regular import descriptor tables. Note that the loader is checking for bounds imports first - an app which runs but doesnt have an import directory may have bound imports instead.

Next LdrpLoadImportModule constructs a Unicode string for each DLL found in the Import Directory and then employs LdrpCheckForLoadedDll to see if they have already been loaded.

Next the LdrpSnapIAT routine examines every DLL referenced in the Import Directory for a value of -1 (i.e again checks for bound imports first). It then changes the memory protection of the IAT to PAGE_READWRITE and proceeds to examine each entry in the IAT before moving on to LdrpSnapThunk Routine.

LdrpSnapThunk uses a function's ordinal to locate its address and determine whether or not its forwarded. Otherwise it calls LdrpNameToOrdinal which uses the binary search on the export table to quickly locate the ordinal. If the function is not found it returns STATUS_ENTRYPOINT_NOT_FOUND, otherwise it replaces the entry in the IAT with the APIs entry point and returns to LdrpSnapIAT which restores the memory protection it changed at the beginning of its work, calls NtFlushInstructionCache to force a cache refresh on the memory block containing the IAT, and returns back to LdrpWalkImportDescriptor.

Win2k insists that ntdll.dll is either loaded as a bound import or in the regular import directory before allowing it to load, whereas win9x allows an app with no imports to load.

This is a very brief overview but shows how the loader must examine evert imported API in order to calculate a real address in memory and to see if an API is being forwarded. Each imported DLL may bring in additional modules and the process will be repeated over and over again until all dependencies have been checked.

In the next post, we see how to navigate imports on the disk.

Dissecting the PE File Format - 7

The Import Section

This section contains info about all functions imported by the executable from DLLs. This is stored in various data structures and the most important of those being the import directory and the import address table. You might also have bound_import  and Delay_Import directories.

The Windows loader is responsible for loading all the DLLs that the application uses and mapping them into the process address space. It has to find the addresses of all the imported functions in their various DLLs and make them available for the executable being loaded.

The addresses of functions inside a DLL are not static but change when DLLs are updated. To account for this, the IAT was used. This is a table of pointers to the function addresses which is filled in the windows loader as the DLLs are loaded.

By using a pointer table, the loader does not need to change the addresses of imported functions everywhere in the code they are called. It only has to correct the addresses in the import table.

The Import Directory

It is an array of IMAGE_IMPORT_DESCRIPTOR structures. Each structure is 20 bytes and contains information about the DLL which our PE file imports functions from. The number of structures is equal to the number of DLLs the PE file imports functions from. There's no field indicating the no. of structures and the last structure is filled with all 0's.

You can find the Import Directory by looking at the Data Directory (80 bytes from the beginning of the PE header)

The first member - OriginalFirstThink which is a DWORD union, may have at one time been a set of flags.  However, Microsoft changed its meaning but did not update winnt.h. This field really contains the RVA of an array of IMAGE_THUNK_DATA structures.

ASIDE - Union is just a redefinition of the same area of memory. The union above does not contain 2 DWORDs  but only one which could either be OriginalFirstThink or Characteristics

The TimeDateStamp is 0 unless the executable is bound when it contains -1.

The ForwarderChain was used for old-style binding and not considered here.

Name1 contains a pointer (RVA) to the ascii name of the DLL.

The last member FirstThunk, also contains the RVA of an array of DWORD-sized IMAGE_THINK_DATA structures. - a duplicate of the first array. If the function described is a bound import (explained below), then FirstThunk contains the actual address of the function instead of an RVA to an IMAGE_THUNK_DATA. These structures are defined as:

Each IMAGE_THUNK_DATA is a DWORD union which effectively has only one of the two values. In the file on disk it either contains the ordinal of the imported function or the RVA to an IMAGE_IMPORT_BY_NAME structure. Once loaded, the ones pointed at by FirstThunk are overwritten  with addresses if imported functions - this becomes the IAT.

Each IMAGE_IMPORT_BY_NAME structure is defined as follows:

Hint - contains the index into the EAT of the DLL the functions resides in. This field is for use by the PE loader so it can look up the functions in the DLLs EAT quickly. The name at that index is tried and if it doesn't match then a binary search is done to find the name. For some linkers this value is not essential and is set to 0.

Name1 - contains the name of the imported function and is a null terminated ASCII string.

The most important parts are the imported DLL names and the arrays of IMAGE_THUNK_DATA structures. Each IMAGE_THUNK_DATA structure corresponds to one imported function from the DLL. The arrays pointed to by the OriginalFirstThunk and FirstThunk run parallel and are terminated by a null DWORD. There are separate paris of arrays of IMAGE_THUNK_DATA structures. for each imported DLL.

In other words, there are several IMAGE_IMPORT_BY_NAME structures. You create two arrays, then fill them with RVAs of those IMAGE_IMPORT_BY_NAME structures, so both arrays contain exactly the same values. Now you assign the RVA of the first array to OriginalFirstThunk and the RVA of the second array to FirstThunk.

The number of elements in the OriginalFirstThunk and FirstThunk arrays depend on the number of functions imported from the DLL. e.g. if the PE file imports 10 functions from user32.dll, Name1 in the IMAGE_IMPORT_DESCRIPTOR structure will contain the RVA of the string "user32.dll" and there will be 10 IMAGE_THUNK_DATAs in each array.

The 2 parallel arrays have been called by several different names but the most common are Import Address Table ( for the one pointed by FirstThunk) or Import Lookup Table (pointed by OriginalFirstThunk)

Why are there 2 parallel arrays of pointers to IMAGE_IMPORT_BY_NAME structures?

The Import Name Table is left alone and never modified. The IAT is overwritten by the actual function addresses by the loader. The loader iterates through each pointer in the arrays and finds the address of the function that each structure refers to. The loader then overwrites the pointer to IMAGE_IMPORT_BY_NAME with the function's addresses. The arrays of RVAs in the Import Name Tables remain unchanged so that if the need arises to find the names of imported functions, the PE loader can still find them.

Although the IAT is pointed to by entry number 12 in data directory, some linkers dont set this directory entry and the app will run nevertheless. The loader only uses this to temporarily mark the IATs as read-write during import resolution and can resolve imports without it.

This is how the windows loader is able to overwrite the IATs when it resides in the read-only section. At load time, the system temporarily sets the attribute of the pages containing the imports data to read/write. Once the imports table is initialized the pages are set back to their original attributes.

Calls to imported functions take place via a function pointer in the IAT and can take 2 forms, one more efficient than the other. e.g. imagine the address 00405030 refers to one of the entries in the FirstThunk array thats overwritten by the loader with the address of GetMessage in USER32.DLL

The efficient way to call GetMessage looks like this: 

0040100C CALL DWORD PTR [00405030 ] 

The inefficient way looks like this: 

0040100C CALL [00402200] 

....... 

....... 

00402200 JMP DWORD PTR [00405030] 

The second method achieves the same but with 5 additional bytes of code and takes longer to execute.

Why are calls to imported functions implemented this way?

The compiler cannot distinguish between calls to ordinary functions within the same module and imported functions and emits the same output for both: CALL [XXXXXX]

where XXXXXX has to be an actual code address (not a pointer) to be filled by the linker later. The linker does not know the address of the imported function and so has to supply a substitute chunk of code - the JMP stub seen above.

The optimized form is obtained using the __declspec(dllimport) modifier to tell the compiler that the function resides in a DLL. It will then output CALL DWORD PTR [XXXXXX].

If __declspec(dllimport) has not been used when compiling the executable there will be a whole collection of jump stubs for imported functions located together somewhere in the code. This has been known by various names - "transfer area", "trampoline", "jump thunk table".

Functions Exported By Ordinal Only

When functions are exported by ordinal only, there will be no IMAGE_THUNK_BY_NAME structure for that function in the caller's module. Instead, the IMAGE_THUNK_DATA for that  function contains the ordinal for that function.

Before the executable is loaded, you can tell if the IMAGE_THUNK_DATA structure contains an ordinal or an RVA by looking at the MSB or high bit. If set, then the lower 32 bits are treated as an ordinal value. If clear, the value is an RVA to an IMAGE_IMPORT_BY_NAME.

Bound Imports

When the loader loads a PE file into memory, it examines the import table and loads the required DLLs into the process address space. Then it walks the array pointed at by FirstThunk and replaces the IMAGE_THUNK_DATAs with the real addresses of import functions. If somehow the programmer can predict the addresses of functions correctly, the PE loader doesn't have to fix the IMAGE_THUNK_DATAs each time the PE file is run as the correct address is already there.

Microsoft compilers come with a utility bind.exe  that examines the IAT of a PE file and replaces the IMAGE_THUNK_DATA dwords with the addresses of imported functions. When the file is loaded, the loader must check i the addresses are valid. If the DLL versions do not match the ones in the PE file or if the DLLs need to be relocated, the loader knows that the bound addresses are stale and it walks the INT to calculate new addresses.

Therefore, although the INT is not necessary for the executable to load, if not present the executable cannot be bound.

The Bound Import Directory

The information the loader uses to determine if bound addresses are valid is kept in a IMAGE_BOUND_IMPORT_DESCRIPTOR structure. A bound executable contains a list of those structures, one for each imported DLL that has been bound:

TimeDateStamp must match the TimeDateStamp of the exporting DLL's FileHeader; if it doesn't match the loader assumes that the binary is bound to a wrong dll and will re-patch the import list.

The OffsetModuleName member contains the offset (not RVA) from the first IMAGE_BOUND_IMPORT_DESCRIPTOR  to the name of the DLL in the null-terminated ASCII.

The NumberOfModuleForwarderRefs member contains the number of IMAGE_BOUND_FORWARDER_REF structures that immediately follow this structure.

Its almost similar to the previous structure. The reason to have 2 similar structures like this is that when binding against a function which is forwarded to another DLL, the validity of that forwarded DLL has to be checked at load time too. This structure contains the details of those forwarded DLLs.

e.g. the function HeapAlloc in kernel32.dll is forwarded to RtAllocateHeap in ntdll.dll. If we created an app which imports HeapAlloc and used bind.exe on the app, there would be an IMAGE_BOUND_IMPORT_DESCRIPTOR for kernel32.dll followed by an IMAGE_BOUND_FORWARDER_REF for ntdll.dll.

In the next post we talk more about the loader.