Friday, March 30, 2007

IShellExecuteHook is dead

Some products like to oversee certain user activities such as the act of starting a new applications (or opening documents) using the shell (explorer). For this purpose Microsoft invented (and documented/supported) a handy little COM interface known as IShellExecuteHook. The monitoring program registers it on a specific path in the registry and the shell loads and calls it when the user executes something in the shell. Here is a sample ATL class with the obvious code omitted:


class ATL_NO_VTABLE
MyShellExecuteHook : public IShellExecuteHook
{
public:
// Provide implementation for IUnknown::QueryInterface
STDMETHOD(QueryInterface)(REFIID riid, void** ppvObject)
// Stock implementation of AddRef and Release works
_ATL_ADDREF_RELEASE_IMPL(MyShellExecuteHook)

// IShellExecuteHook has just one method
STDMETHOD(Execute)(LPSHELLEXECUTEINFO lpsei)
{ your code here }
};


The beauty of IShellExecuteHook is that it allows you to pass along any requests that you don't care about to the shell. You are basically an observer with the right to veto (block) anything you don't like.

In Vista, the IShellExecuteHook API has been deprecated. There is a workaround involving editing a couple of undocumented registry entries but I won't go in there since the message is loud and clear: do not use this.

However, if you ask a lot Microsoft support will coff a workaround which is to take over the open command by changing the registry like so:

REGEDIT4

[HKEY_CURRENT_USER\Software\Classes\Folder]
@=""

[HKEY_CURRENT_USER\Software\Classes\Folder\shell]

[HKEY_CURRENT_USER\Software\Classes\Folder\shell\open]

[HKEY_CURRENT_USER\Software\Classes\Folder\shell\open\command]
@="path to your exe here"


Now, if you do this then you have to take over the interaction with all shell namespace extensions, including third party extensions (specially the ones that use IShellFolder).

Of course this is insane. So scratch that idea.

A better idea would be to register another verb such as 'open with xxxx' and make it the default verb (the default verb handles the double-click action). Yes, I hear you; that is not the issue that was solved by IShellExecuteHook.

At this point in the game people that were relying on this documented way to intercept the shell and do something unique for special cases but for everything else fallback to let the shell do it's thing are left with no option. And when people are left with no documented option but still have to ship a product they start reverse engineering the shell. It turns out that the shell itself has a special magic component that takes over for 'open'. Meet the DelegateExecute.

The DelegateExecute Value in HKCR\Folder\shell\open\command is a CLSID and if you put your own COM object there you get queried for ICommandExecute interface. I have not verified this myself nor I am interested in cooking an unsupported way to observe the shell actions. My point was a) Microsoft giveth and Microsoft can taketh away and b) people will find a way to get it back by force if necessary.

Such are the wonderful dynamics of developing for Windows.

Wednesday, March 28, 2007

A funny question from my bank

Yesterday I logged on to my bank account and applied for a service. After the second or third page of the application form the website lobbed this question to me:
Are you a senior foreign political figure or a family member or associate of a senior foreign political figure?

What??

Mmmm.. I guess it not something you want to answer in the positive to get the VIP-level service. Now come on .. seriously, what does it mean to be an 'associate' and what qualifies as 'senior'.
The other interesting fact is that I applied a couple of months ago for the very same service, different bank (different account) in person. The service desk lady asked me all the standard questions and most certainly did not ask that one. I guess this is either a) something that a bank employee can tell just by looking at the person or b) my other bank has gone bananas.

If I had to choose, I'll go with a) and that means that the lady selected 'yes' (selecting 'no' will be worse for her) and that means that I'll get a visit from the secret service (or the FBI) any day now.

On a more serious note, after some research I can tell you where this nonsense comes from. It is the implementation of section 312 of Title III of the Patriot Act of 2001 that "requires financial institutions to determine whether any holder of a private banking account is a senior foreign political figure or a family member or close personal associate of such person and to report any known or suspected violation of law conducted through or involving the account". So how do you do that if you are not a private investigator? Simple, just ask the customer.

So I guess who is going to pay me a visit is more likely the department of homeland security.

Wednesday, March 14, 2007

Pattern matching HTML

One sometimes stumbles across web security systems where at the core an HTML pattern matching engine lives. The snake oil alarm should ring at this time. Filtering or pattern matching HTML for any security purpose is incredibly hard (if not impossible) to do. Take for example the following html snippet:

<BDO DIR="rtl">rat</BDO>'.'<BDO DIR="rtl">parc<HRM></BDO>

Can anybody just by looking at it tell me what would a browser display? If if you can; then tell if I change one single character of the above what would you see in a browser?

<BDO DIR="rtl">rat</BDO>'o'<BDO DIR="rtl">parc<HRM></BDO>

Part of the problem resides in the fact that most browsers are purposely lax when parsing html because a lot of pages out there were broken, i.e did not have valid (compliant) html and the burden was put on the browsers to show them. For example, it is not a problem to forget the HTML or BODY tag as the previous examples show.

The other reason is that HTML is trying to be too many things at once because it got too popular too fast and many special interest groups decided to piggyback their needs on top of it. This has resulted in a (markup) language that is like an art piece: open to interpretation.

All that and we haven't even begin to talk about DHTML. So if you are considering some regex engine for your next web security product, please, pretty please don't.

Monday, March 12, 2007

VS Debugger Kung-Fu: Pseudo Variables

In VS2005 while debugging you can view certain variables that are not explicitly defined by your code but defined by the OS enironment. People call them pseudo variables and the official doucmentation is here (msdn) for example here are a couple of the least well known of them:

$user : shows the process token information
$tib : points to the tread information block (TIB) structure (same as FS register)
$vframe : shows you the address of the current stack frame (sometimes in EBP)

From these three the $user is the one that is easiest to grok. It shows you the principal domain\name, the SID, the impersonation level, the priviledges, groups and tells you if the current thread is impersonating. This is very neat except that in a typical VS debugging scenario there is rarely a doubth that you are running as a very priviledged user.

But the one I want to talk about is the $tib one. The Tread information block is a key user-mode structure for windows. Every C or C++ compiler for Windows generates code that uses it. Here is the structure as declared in WinNT.h:
typedef struct _NT_TIB
{
_EXCEPTION_REGISTRATION_RECORD *ExceptionList;
PVOID StackBase;
PVOID StackLimit;
PVOID SubSystemTib;
union {
PVOID FiberData;
DWORD Version;
};
PVOID ArbitraryUserPointer;
struct _NT_TIB *Self;
} NT_TIB;

The TIB is actually the tip of the iceberg; the first part of a larger structure known as the TEB. This structure is so important that in user-mode the FS register always points to it therefore the structure offsets can be used as offsets into the segment pointed to by the FS register. For example, FS:[0] points to the ExceptionList which is the head of structured exception handling chain.

Just bear in mind that each thread has a different TIB so make sure you use the right structure. You can view the current thread structure members by using this incantation on a VS watch window:
(NT_TIB*)$tib

By taking note of the value of the StackBase and StackLimit you can know what is the range of addresses that are normal stack values. Note that StackLimit shows the lowest 'committed' page of the stack. As your program uses more of the thread stack this value will change to lower values. So whenever you a piece of data that you don't recognize you know have a way to find out if it belongs to one of your stacks or belongs to some heap.

For example, for a plain single threaded application it is normal to see the stack base at 0x00130000 which is just 1.1 Mb above the 0 (zero) address, and everybody knows that the default stack size of a win32 thread is 1 Mb, yet you normally see the stack limit being an address closer to the above value than to zero. In fact if you see the stack limit anywhere close 0x00030000 you are having serious stack overflow problems.

Now let's talk about ExceptionList. This member points to the current head of the structured exception chain, an undocumented but well known structure:

typedef struct _EXCEPTION_REGISTRATION_RECORD
{
_EXCEPTION_REGISTRATION_RECORD * pNext;
FARPROC pfnHandler;
} EXCEPTION_REGISTRATION_RECORD


The pNext member (confusingly) points to the previous head of the exception chain while the pfnHandler is the address of function that will be called in case of an exception.

Here is all the code that I am using as reference:

int APIENTRY wWinMain(HINSTANCE, HINSTANCE, LPTSTR, int)
{

MSG msg;
while(GetMessage(&msg, NULL, 0, 0))
{
TranslateMessage(&msg);
DispatchMessage(&msg);
}

return (int) msg.wParam;
}


In my home computer with a breakpoint in the start of winMain(), (next to the while statement) the stack base is 0x00130000 and the stack limit value is 0x0012d000. The value of ExceptionList is 0x0012ffb0 which is normal and means that the exception record lives very close to the top of the stack. If we dump that memory location we find the exception record:

0x0012FFB0: 0012ffe0 00401f24

Which means that the exception handling routine is at 0x00401f24, which is within our EXE (check the previous blog if you don't know how I know this) and that there is a previous exception record at 0x0012ffe0 also inside the stack (which is a good sanity-check). If we dump that in turn we see:

0x0012FFE0: ffffffff 7c839aa8

And given the 'all ones' value it is obvious there is no previous exception record.

What this all means is that the code in main is executing under two SEH exception frames or in other words it is executing under two nested __try{ }__except( ) blocks. But wait! I don't have SEH in my code?? so how come I have two exception frames? who put them there? why?

By the way (and before we solve the mistery), the method by which these structures are built is a fairly common assembler pattern that you will see a lot when you step thru code and it goes something like this:


PUSH
PUSH FS:[00000000]
MOV DWORD PTR FS:[00000000],ESP


The two push instructions create the new exception registration record in the stack and the mov instruction makes it the new head of the exception chain. So what you just saw was the code that implements the __try part.

Ok, going back to the question the answer is simple: the wWinMain function is not the first function to execute from my EXE; since I compiled with the standard C runtime support, the real entry point to the application is wWinMainCRTStartup(void) which you can find in the VS directories in a file called wwincrt0.c (for VS 2003). If you look at this function you will see that in essense does this:

int wWinMainCRTStartup()
{
int mainret;
//set SEH frame
__try
{
//use win32 API to retrieve the params of winmain
//then call:
mainret = wWinMain(h, NULL, cl, f)
}
__except (_XcptFilter(GetExceptionCode(),
GetExceptionInformation()))
{
mainret = GetExceptionCode();
}
return mainret;
}


This accounts for one SEH frame (the inner one) and the other one is placed by the OS when the thread was created. This outer handler lives as one might expect in kernel32.dll. Ah, one final note about SEH; if you are using C++ try{ }catch{ } they also use SEH under the covers, but of course the inner workings of that are way outside the scope of this post.

So my original point of this post was to show you how to use $tib to find the stack location and to discover the SEH frames that are in scope. The obvious trick here is that if you are debugging strange, magical or just plain wierd behavior on a thread it is not a bad idea to put some breakpoints in the exception handlers and now you know how to find them.

Tuesday, March 6, 2007

Windows 32-bit Memory Map

Debugging under Windows (even if you made the program that you are debugging) can feel like an expedition to a the amazon jungle; it is very easy to get lost. A thing that is known to help is to have a map and a compass. In case you actually don't visit jungles but like to explore crashdumps, spelunk with breakpoints or climb asserts then the following memory map of Windows would be of interest. This memory map is for 32 versions of Windows such as Windows XP and Windows Server 2003. At the end I have some notes & caveats that you might want to read.







Kernel


0xffffffff

Last virtual address


0xffdff120

Processor 0 Control Block (PRCB)


0xffdff000

Processor 0 Control Region (PCR)

KUSER

SHARED

DATA

0xffdf0300

Syscall stub pointer. Used like this:

mov edx, 0x7ffe0300; call dword ptr [edx]

0xffdf0000

SharedUserData start mapped RW (see 0x7ffe0000)


0xffc00000

Reserved for HAL usage


0xffbfffff

Crash dump driver area end

0xffbe0000

Crash dump driver area


0xffbdffff

End of system non-paged pool area








0xe1000000

Start of system paged area








0xc0c00000

System cache structures and working set list


0xc0800000

System cache begin address


0xc0400000

Process and hyperspace working set lists

Page tables

0xc03fffff


0xc0300000

Current process page directory (1024 entries) is mapped here. The CR3 points to same phys page

0xc0000000

Here is where the OS puts all the page tables for all processes


0xbc3fffff

Session paged pool end

0xbc000000

Session paged pool start




Kernel and HAL code

0x80a6b000

Ntoskernel.exe end address W2k3 SP1

0x80800000

Ntoskernel.exe base address W2k3 SP1

0x806eb780

Ntoskernel.exe end address XP SP2

0x804d7000

Ntoskernel.exe base address XP SP2

0x80036400

Interrupt descriptor table (IDT) 256 entries

0x80036000

Global descriptor table (GDT) 128 entries

0x80000000

Lowest kernel mode address



User


0x7fffffff

Highest user mode address

KUSER SHARED DATA

0x7ffe0300

Syscall stub pointer. Used like this:

mov edx, 0x7ffe0300; call dword ptr [edx]

0x7ffe02d8

TS Active console Id

0x7ffe0000

SharedUserData start (mapped RO)



Heap handles PEB.ProcessHeaps

PEB

0x7ffdf000

Process environment block (PEB) <notXPSP2>

TEB & TIB zone

0x7ffde000

1st thread environment block (TEB) 4kb

0x7ffdd000

2nd thread environment block (TEB) 4kb

0x7ffdc000

3rd thread environment block (TEB) 4kb




Core System DLLs


usual base address

0x7c8d0000

shell32.dll

0x7c800000

ntdll.dll

0x77f50000

advapi32.dll

0x77e40000

kernel32.dll

0x77c50000

rpcrt4.dll

0x77c00000

gdi32.dll

0x77670000

ole32.dll

0x77420000

comctl32.dll

0x77380000

user32.dll

Heaps

0x00b80000

Heap 3 (sample alloc 1MB)

0x00a70000

Heap 2 (sample alloc 1MB)

0x00960000

Heap 0 (sample alloc 1MB)

EXE


Usual base address


.rsrc section (icons, manifest)


.data section


.rdata section (Import address table (IAT))

0x00401000

.text section

0x00400000

PE header = ‘MZ’




Text support tables

0x00310000

Start of sorttbls.nls

0x002C0000

Start of sortkey.nls

0x00280000

Start of locale.nls (or 0x001c0000)

0x00260000

Start of Unicode.nls (or 0x001a0000)




Heaps

0x00250000

Heap 1 (sample alloc 16kB)

0x00150000

Heap 0 (sample alloc 64KB)




Stack of first thread

0x00130000

Stack base (typical)

0x0012ffe0

Typical location of first SEH frame

0x0012ffa8

Typical location of second SEH frame

0x0012ff08

EBP value in WinMain( ) entry

0x00126000

End of initially allocated stack

0x00030100

Typical stack end (~1MB) Typically not mapped


0x0002100A

Memory not mapped (invalid)


0x00020b00

Approximate location of lpCommandline



Memory not mapped (invalid)


0x00020012

Strings (paths?)


0x00012000

Memory not mapped (invalid)


0x00010000

Approximate location of the environment block



Memory not mapped (invalid)

0x00000000

Memory not mapped (invalid)




Notes:
1 - Every version and service pack of Windows changes in some little way the location of some elements but in general things are still to be found in the approximate location shown above. XP SP2 introduces some intentional randomness in the user-mode memory map as a security measure.

2 - The User Mode side of the memory map assumes that you have an EXE loaded and running and that the EXE links with most of the usual Windows DLLs such as User32.dll and so on.

3 - Notice that I have put the approximate locations of four (4) user mode heaps! Until not too long ago I assumed that most programs would have one or at most two heaps; this is not the case in my computer and might be an artifact of other applications that like to inject DLLs in other processes.

4 - While everybody knows that user mode applications get 2GB to play with, note that the way the memory is structured makes it difficult to use it as contiguous chunk: system DLLs grow from the topmost user-mode address towards zero which means that you probably won't be able to use anything above 0x77000000 and on the low end the EXE is usually located at 0x00400000 and below it you will tipically find the unicode support tables (memmory mapped) and the stacks and below the stacks other stuff placed by Windows. This means that the biggest contiguous chunk of RAM available will be about 1.8GB, or in other words 200MB are lost given the inherent fragmentation of user-mode memory.