Read the latest news and views from the OpenCandy team.

CandyWrap #1: Process spawning, RunDll32, & the PATH variable

Welcome to CandyWraps!

This is the first in a series of articles intended to help application developers and publishers better understand our system and get the most out of their relationship with OpenCandy.

These posts will serve as our means of sharing the knowledge and experience we’ve gained from building and running OpenCandy products and technologies. This includes everything from PHP and MySQL on the server-side to Win32 APIs and installer platforms on the client-side to designing intuitive product installers and software. The topics covered, amount of detail and level of technicality will vary from post to post and may include everything from general overviews and development-related anecdotes to detailed designs and source code.

The subject of today’s CandyWrap is intended for developers, and is a small but useful tip about executing RunDll32.exe by way of a Windows process.

Very quickly, the tip is: Don’t forget about the PATH environment variable when using Windows process APIs, such as _spawn(). For the details read on…

Say, for example, that you want to call a function that is exported from an unmanaged DLL. The safest way to do so is to use the process APIs provided by Visual Studio’s Runtime libraries, specifically _spawnl().

This is what it looks like:

intptr_t _spawnl(int mode, const char *cmdname, const char *arg0, const char
*arg1, ... const char *argn, NULL );

MSDN _spawn() API reference: http://msdn.microsoft.com/en-us/library/20y988d2(VS.71).aspx

Now, the following example displays a standard Windows message box with the text “HelloWorld!”. It’s done by executing RunDLL32.exe and calling the MessageBoxA function from user32.dll.

intptr_t ret=_spawnlp(_P_NOWAIT, "RunDll32.exe","RunDll32.exe",
"user32.dll,MessageBoxA HelloWorld! ", 0);

The only problem is no message box appears. After monitoring the system you determine the MessageBoxA function isn’t being called so you check the usual suspects:

  • Is the process being spawned?
  • Does RunDll32.exe crash?
  • Is there something wrong with the MessageBoxA function or the way it’s being called?
  • Is there something wrong with the loading of user32.dll? LoadLibrary()fail?

They all look fine, so what could be wrong? Chances are the PATH environment variable doesn’t include a reference to the System32 directory (%SYSTEMROOT%\SYSTEM32), preventing _spawnl() from locating RunDll32.exe!

The lesson? Don’t always expect relative paths to be resolved automatically. In this case, using GetSystemWindowsDirectory() to build an absolute path to RunDll32.exe is well worth the trouble.

That’s it for this CandyWrap. Stay tuned to this channel for more information, tips and hints. See you next time.

Adrian Bourke is a handyman at OpenCandy working on everything. Check out Adrian’s bio.

Creative Commons License
This post is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.

3 Comments so far
  1. by John Kelley / March 31, 2010

    Can you explain your choice of _spawn over the traditional LoadLibrary/GetProcAddress?

    It’s rather strange to spawn a new process just to call a DLL export unless you’re being extremely paranoid and don’t want the DLL being loaded into your address space.

  2. by Adrian / March 31, 2010

    John,

    Good question. This is done because the DLL needs to be capable of running even if the calling process ends, ie. asynchronously and in it’s own process.

  3. by CandyPicks #1: Rapid Environment Editor, freeware Windows environment variables editor | Helping Software Developers and Users - OpenCandy / April 9, 2010

    [...] week ago, my colleague Adrian wrote a CandyWrap article intended primarily for developers about Rundll32.exe and the system PATH environment [...]

3 Responses to “CandyWrap #1: Process spawning, RunDll32, & the PATH variable”

By submitting a comment here you grant OpenCandy a perpetual license to reproduce your words and name/web site in attribution. Inappropriate comments will be removed at admin's discretion.