Friday, March 30, 2007

IShellExecuteHook is dead

Some products like to oversee certain user activities such as the act of starting a new applications (or opening documents) using the shell (explorer). For this purpose Microsoft invented (and documented/supported) a handy little COM interface known as IShellExecuteHook. The monitoring program registers it on a specific path in the registry and the shell loads and calls it when the user executes something in the shell. Here is a sample ATL class with the obvious code omitted:


class ATL_NO_VTABLE
MyShellExecuteHook : public IShellExecuteHook
{
public:
// Provide implementation for IUnknown::QueryInterface
STDMETHOD(QueryInterface)(REFIID riid, void** ppvObject)
// Stock implementation of AddRef and Release works
_ATL_ADDREF_RELEASE_IMPL(MyShellExecuteHook)

// IShellExecuteHook has just one method
STDMETHOD(Execute)(LPSHELLEXECUTEINFO lpsei)
{ your code here }
};


The beauty of IShellExecuteHook is that it allows you to pass along any requests that you don't care about to the shell. You are basically an observer with the right to veto (block) anything you don't like.

In Vista, the IShellExecuteHook API has been deprecated. There is a workaround involving editing a couple of undocumented registry entries but I won't go in there since the message is loud and clear: do not use this.

However, if you ask a lot Microsoft support will coff a workaround which is to take over the open command by changing the registry like so:

REGEDIT4

[HKEY_CURRENT_USER\Software\Classes\Folder]
@=""

[HKEY_CURRENT_USER\Software\Classes\Folder\shell]

[HKEY_CURRENT_USER\Software\Classes\Folder\shell\open]

[HKEY_CURRENT_USER\Software\Classes\Folder\shell\open\command]
@="path to your exe here"


Now, if you do this then you have to take over the interaction with all shell namespace extensions, including third party extensions (specially the ones that use IShellFolder).

Of course this is insane. So scratch that idea.

A better idea would be to register another verb such as 'open with xxxx' and make it the default verb (the default verb handles the double-click action). Yes, I hear you; that is not the issue that was solved by IShellExecuteHook.

At this point in the game people that were relying on this documented way to intercept the shell and do something unique for special cases but for everything else fallback to let the shell do it's thing are left with no option. And when people are left with no documented option but still have to ship a product they start reverse engineering the shell. It turns out that the shell itself has a special magic component that takes over for 'open'. Meet the DelegateExecute.

The DelegateExecute Value in HKCR\Folder\shell\open\command is a CLSID and if you put your own COM object there you get queried for ICommandExecute interface. I have not verified this myself nor I am interested in cooking an unsupported way to observe the shell actions. My point was a) Microsoft giveth and Microsoft can taketh away and b) people will find a way to get it back by force if necessary.

Such are the wonderful dynamics of developing for Windows.

No comments: