Sunday, September 7, 2008

Element 24 is out

Yes! finally Google Chrome and Chromium have been launched and I can talk about what I have been doing. Unless you have been in a deep comma you should have seen the news about it and you might have already tried it. In my humble opinion it has been well received, even by many skeptics and even a few praises by people that are well versed in windows which are MSFT fans.

From all the stuff that goes in a browser I have been most involved with the Sandbox, the first-run experience and with the installer. I'll talk about them in due time. Of course, in many cases it would be somewhat peripherial because I rather improve the official documents that make my blog (that at last count had two subscribers) the place to go to learn about them.

By far to me the most interesting thing is the Sandbox but I haven't seen much talk about it on the security and rev-engineering sites that I monitor. In the RE community I guess is not that interesting since it is open source, so not much to reverse there. The system-level security community still silent, I think they are digesting it.

Anyhow, here is the obligatory link to the design documents. The sandbox design doc is there.

Also, Nic is also going to talk about the sandbox.

Saturday, July 14, 2007

Long time no post

So I haven't written anything in a while. Since mid-May I have been working on a large software company (the small company I used to work for got acquired by this other big company) which has many interesting but secret projects. And secrecy is a big thing for my new master.

Therefore me blogging about software seems kind of dangerous. Anyhow, is not like there are people reading this blog anyway.

This is more like a note in case somebody finds this blogs and wonders what happened. Did he got abducted by aliens? a CIA 'wet' operation?

Nope. I just got a job.

Wednesday, April 4, 2007

Memory latency

I recently found someone that measured (circa 2006) the different latencies of the memory hiearchy on a typical PC. Here are the important numbers:
levellatency
L1 cache (on die)1.4ns
L2 cache (off die)9.7ns
RAM (PC2700 DDR) 28.5ns
Hard Drive (250 GB Maxtor 7200 RPM) 25.6ms

The test machine had an AMD Athlon XP 3000+ processor. Another interesting piece of data is the throuput (sustained data transfer) of the RAM, 2541 MB/s versus of the hard disk: 67 MB/s.

Note that in Intel x86 and AMD processors besides the hiearchy of cache, you also have the translation lookaside buffer (aka TLB) which is a small cache that keeps the most recent virtual-to-physical translations. This quite important if you code runs in an OS that uses virtual memory such as Windows or Linux or Solaris or MacOsX. When there is a TLB miss the CPU has to scan the page tables (living in RAM) to find the appropiate entry. This results in about 3 to 4 RAM reads. In terms of the size, the 1995 Pentium had one code TLB and two data TLBs, one of the data TLB has 64 entries (for 4Kb pages) and the other 8 entries (for 4Mb pages). The instruction TLB is 32 entries.

The biggest hit in terms of TLB-related (or virtual-to-physical translation) happens when the OS switches processes. This causes the active page tables to change which in turn requires a total (or partial) TLB invalidation.

The credits for the measures go to Diomidis Spinellis in his paper "Some Types of Memory are More Equal than Others".

Friday, March 30, 2007

IShellExecuteHook is dead

Some products like to oversee certain user activities such as the act of starting a new applications (or opening documents) using the shell (explorer). For this purpose Microsoft invented (and documented/supported) a handy little COM interface known as IShellExecuteHook. The monitoring program registers it on a specific path in the registry and the shell loads and calls it when the user executes something in the shell. Here is a sample ATL class with the obvious code omitted:


class ATL_NO_VTABLE
MyShellExecuteHook : public IShellExecuteHook
{
public:
// Provide implementation for IUnknown::QueryInterface
STDMETHOD(QueryInterface)(REFIID riid, void** ppvObject)
// Stock implementation of AddRef and Release works
_ATL_ADDREF_RELEASE_IMPL(MyShellExecuteHook)

// IShellExecuteHook has just one method
STDMETHOD(Execute)(LPSHELLEXECUTEINFO lpsei)
{ your code here }
};


The beauty of IShellExecuteHook is that it allows you to pass along any requests that you don't care about to the shell. You are basically an observer with the right to veto (block) anything you don't like.

In Vista, the IShellExecuteHook API has been deprecated. There is a workaround involving editing a couple of undocumented registry entries but I won't go in there since the message is loud and clear: do not use this.

However, if you ask a lot Microsoft support will coff a workaround which is to take over the open command by changing the registry like so:

REGEDIT4

[HKEY_CURRENT_USER\Software\Classes\Folder]
@=""

[HKEY_CURRENT_USER\Software\Classes\Folder\shell]

[HKEY_CURRENT_USER\Software\Classes\Folder\shell\open]

[HKEY_CURRENT_USER\Software\Classes\Folder\shell\open\command]
@="path to your exe here"


Now, if you do this then you have to take over the interaction with all shell namespace extensions, including third party extensions (specially the ones that use IShellFolder).

Of course this is insane. So scratch that idea.

A better idea would be to register another verb such as 'open with xxxx' and make it the default verb (the default verb handles the double-click action). Yes, I hear you; that is not the issue that was solved by IShellExecuteHook.

At this point in the game people that were relying on this documented way to intercept the shell and do something unique for special cases but for everything else fallback to let the shell do it's thing are left with no option. And when people are left with no documented option but still have to ship a product they start reverse engineering the shell. It turns out that the shell itself has a special magic component that takes over for 'open'. Meet the DelegateExecute.

The DelegateExecute Value in HKCR\Folder\shell\open\command is a CLSID and if you put your own COM object there you get queried for ICommandExecute interface. I have not verified this myself nor I am interested in cooking an unsupported way to observe the shell actions. My point was a) Microsoft giveth and Microsoft can taketh away and b) people will find a way to get it back by force if necessary.

Such are the wonderful dynamics of developing for Windows.

Wednesday, March 28, 2007

A funny question from my bank

Yesterday I logged on to my bank account and applied for a service. After the second or third page of the application form the website lobbed this question to me:
Are you a senior foreign political figure or a family member or associate of a senior foreign political figure?

What??

Mmmm.. I guess it not something you want to answer in the positive to get the VIP-level service. Now come on .. seriously, what does it mean to be an 'associate' and what qualifies as 'senior'.
The other interesting fact is that I applied a couple of months ago for the very same service, different bank (different account) in person. The service desk lady asked me all the standard questions and most certainly did not ask that one. I guess this is either a) something that a bank employee can tell just by looking at the person or b) my other bank has gone bananas.

If I had to choose, I'll go with a) and that means that the lady selected 'yes' (selecting 'no' will be worse for her) and that means that I'll get a visit from the secret service (or the FBI) any day now.

On a more serious note, after some research I can tell you where this nonsense comes from. It is the implementation of section 312 of Title III of the Patriot Act of 2001 that "requires financial institutions to determine whether any holder of a private banking account is a senior foreign political figure or a family member or close personal associate of such person and to report any known or suspected violation of law conducted through or involving the account". So how do you do that if you are not a private investigator? Simple, just ask the customer.

So I guess who is going to pay me a visit is more likely the department of homeland security.

Wednesday, March 14, 2007

Pattern matching HTML

One sometimes stumbles across web security systems where at the core an HTML pattern matching engine lives. The snake oil alarm should ring at this time. Filtering or pattern matching HTML for any security purpose is incredibly hard (if not impossible) to do. Take for example the following html snippet:

<BDO DIR="rtl">rat</BDO>'.'<BDO DIR="rtl">parc<HRM></BDO>

Can anybody just by looking at it tell me what would a browser display? If if you can; then tell if I change one single character of the above what would you see in a browser?

<BDO DIR="rtl">rat</BDO>'o'<BDO DIR="rtl">parc<HRM></BDO>

Part of the problem resides in the fact that most browsers are purposely lax when parsing html because a lot of pages out there were broken, i.e did not have valid (compliant) html and the burden was put on the browsers to show them. For example, it is not a problem to forget the HTML or BODY tag as the previous examples show.

The other reason is that HTML is trying to be too many things at once because it got too popular too fast and many special interest groups decided to piggyback their needs on top of it. This has resulted in a (markup) language that is like an art piece: open to interpretation.

All that and we haven't even begin to talk about DHTML. So if you are considering some regex engine for your next web security product, please, pretty please don't.

Monday, March 12, 2007

VS Debugger Kung-Fu: Pseudo Variables

In VS2005 while debugging you can view certain variables that are not explicitly defined by your code but defined by the OS enironment. People call them pseudo variables and the official doucmentation is here (msdn) for example here are a couple of the least well known of them:

$user : shows the process token information
$tib : points to the tread information block (TIB) structure (same as FS register)
$vframe : shows you the address of the current stack frame (sometimes in EBP)

From these three the $user is the one that is easiest to grok. It shows you the principal domain\name, the SID, the impersonation level, the priviledges, groups and tells you if the current thread is impersonating. This is very neat except that in a typical VS debugging scenario there is rarely a doubth that you are running as a very priviledged user.

But the one I want to talk about is the $tib one. The Tread information block is a key user-mode structure for windows. Every C or C++ compiler for Windows generates code that uses it. Here is the structure as declared in WinNT.h:
typedef struct _NT_TIB
{
_EXCEPTION_REGISTRATION_RECORD *ExceptionList;
PVOID StackBase;
PVOID StackLimit;
PVOID SubSystemTib;
union {
PVOID FiberData;
DWORD Version;
};
PVOID ArbitraryUserPointer;
struct _NT_TIB *Self;
} NT_TIB;

The TIB is actually the tip of the iceberg; the first part of a larger structure known as the TEB. This structure is so important that in user-mode the FS register always points to it therefore the structure offsets can be used as offsets into the segment pointed to by the FS register. For example, FS:[0] points to the ExceptionList which is the head of structured exception handling chain.

Just bear in mind that each thread has a different TIB so make sure you use the right structure. You can view the current thread structure members by using this incantation on a VS watch window:
(NT_TIB*)$tib

By taking note of the value of the StackBase and StackLimit you can know what is the range of addresses that are normal stack values. Note that StackLimit shows the lowest 'committed' page of the stack. As your program uses more of the thread stack this value will change to lower values. So whenever you a piece of data that you don't recognize you know have a way to find out if it belongs to one of your stacks or belongs to some heap.

For example, for a plain single threaded application it is normal to see the stack base at 0x00130000 which is just 1.1 Mb above the 0 (zero) address, and everybody knows that the default stack size of a win32 thread is 1 Mb, yet you normally see the stack limit being an address closer to the above value than to zero. In fact if you see the stack limit anywhere close 0x00030000 you are having serious stack overflow problems.

Now let's talk about ExceptionList. This member points to the current head of the structured exception chain, an undocumented but well known structure:

typedef struct _EXCEPTION_REGISTRATION_RECORD
{
_EXCEPTION_REGISTRATION_RECORD * pNext;
FARPROC pfnHandler;
} EXCEPTION_REGISTRATION_RECORD


The pNext member (confusingly) points to the previous head of the exception chain while the pfnHandler is the address of function that will be called in case of an exception.

Here is all the code that I am using as reference:

int APIENTRY wWinMain(HINSTANCE, HINSTANCE, LPTSTR, int)
{

MSG msg;
while(GetMessage(&msg, NULL, 0, 0))
{
TranslateMessage(&msg);
DispatchMessage(&msg);
}

return (int) msg.wParam;
}


In my home computer with a breakpoint in the start of winMain(), (next to the while statement) the stack base is 0x00130000 and the stack limit value is 0x0012d000. The value of ExceptionList is 0x0012ffb0 which is normal and means that the exception record lives very close to the top of the stack. If we dump that memory location we find the exception record:

0x0012FFB0: 0012ffe0 00401f24

Which means that the exception handling routine is at 0x00401f24, which is within our EXE (check the previous blog if you don't know how I know this) and that there is a previous exception record at 0x0012ffe0 also inside the stack (which is a good sanity-check). If we dump that in turn we see:

0x0012FFE0: ffffffff 7c839aa8

And given the 'all ones' value it is obvious there is no previous exception record.

What this all means is that the code in main is executing under two SEH exception frames or in other words it is executing under two nested __try{ }__except( ) blocks. But wait! I don't have SEH in my code?? so how come I have two exception frames? who put them there? why?

By the way (and before we solve the mistery), the method by which these structures are built is a fairly common assembler pattern that you will see a lot when you step thru code and it goes something like this:


PUSH
PUSH FS:[00000000]
MOV DWORD PTR FS:[00000000],ESP


The two push instructions create the new exception registration record in the stack and the mov instruction makes it the new head of the exception chain. So what you just saw was the code that implements the __try part.

Ok, going back to the question the answer is simple: the wWinMain function is not the first function to execute from my EXE; since I compiled with the standard C runtime support, the real entry point to the application is wWinMainCRTStartup(void) which you can find in the VS directories in a file called wwincrt0.c (for VS 2003). If you look at this function you will see that in essense does this:

int wWinMainCRTStartup()
{
int mainret;
//set SEH frame
__try
{
//use win32 API to retrieve the params of winmain
//then call:
mainret = wWinMain(h, NULL, cl, f)
}
__except (_XcptFilter(GetExceptionCode(),
GetExceptionInformation()))
{
mainret = GetExceptionCode();
}
return mainret;
}


This accounts for one SEH frame (the inner one) and the other one is placed by the OS when the thread was created. This outer handler lives as one might expect in kernel32.dll. Ah, one final note about SEH; if you are using C++ try{ }catch{ } they also use SEH under the covers, but of course the inner workings of that are way outside the scope of this post.

So my original point of this post was to show you how to use $tib to find the stack location and to discover the SEH frames that are in scope. The obvious trick here is that if you are debugging strange, magical or just plain wierd behavior on a thread it is not a bad idea to put some breakpoints in the exception handlers and now you know how to find them.