7/15/2012

07-15-12 - libpng DLL Delay Load

cblib for a long time has used libpng , which creates a damn DLL dependency. Annoyingly, that happens at app startup, which means even if you are trying to load a .BMP it will fail to start the app if it can't find the PNG DLL (or, you know, using cblib on totally non-image related stuff). What I've wanted for a long time is to put the PNG DLL load into the PNG code so that it's only done if you actually try to load a PNG.

MS could have very easily put DLL load-on-first use into the thunk libs they make. That would have been nice. (ADDENDUM : I guess they did!)

Anyhoo, you can do it yourself, and it looks like this :

We need CALL_IMPORT from before :


template <typename t_func_type>
t_func_type GetImport( t_func_type * pFunc , const char * funcName, HMODULE hm )
{
    if ( *pFunc == 0 )
    {
        ASSERT( hm != 0 );
        t_func_type fp = (t_func_type) GetProcAddress( hm, funcName );
        // not optional :
        ASSERT_RELEASE_THROW( fp != 0 );
        *pFunc = fp;
    }
    return (*pFunc); 
}

#define CALL_IMPORT(name,hm) (*GetImport(&STRING_JOIN(fp_,name),STRINGIZE(name),hm))

and I'm going to just explicitly load the PNG DLL when I get to LoadPNG :

static HMODULE hm_png = 0;
#define HMODULE_FAILED  (HMODULE) 1

static bool my_png_init()
{
    if ( hm_png ) 
    {
        return ( hm_png == HMODULE_FAILED ) ? false : true;
    }
    
    HMODULE hm_zl = LoadLibrary(ZLIB_NAME);
    if ( hm_zl == 0 )
    {
        lprintf("Couldn't load Zlib (%s)\n",ZLIB_NAME);
        hm_png = HMODULE_FAILED;
        return false;
    }
    
    hm_png = LoadLibrary(PNG_NAME);
    if ( hm_png == 0 )
    {
        lprintf("Couldn't load PNG lib (%s)\n",PNG_NAME);
        hm_png = HMODULE_FAILED;
        return false;
    }
    
    lprintf_v2("Using libpng.\n");
                
    return true;
}

#define CALL_PNG(name)      CALL_IMPORT(name,hm_png)

so now we just have to replace all our png calls with CALL_PNG() calls, like :

    png_ptr = png_create_read_struct(PNG_LIBPNG_VER_STRING, NULL,NULL,NULL);

    ->

    png_ptr = CALL_PNG(png_create_read_struct)(PNG_LIBPNG_VER_STRING, NULL,NULL,NULL);

(obviously you could #define png_create_read_struct CALL_PNG(png_create_read_struct) to make it totally transparent)

Lastly we need to define all the fp_ func pointers with the right signature. This is particularly easy because png.h wraps its function externs with macros (nice, thanks guys), so we can just abuse those macros. png.h contains things like :


PNG_EXPORT(png_voidp,png_get_io_ptr) PNGARG((png_structp png_ptr));

So we can just do :

#define PNG_EXPORT(ret,func)    static ret (PNGAPI * STRING_JOIN(fp_,func))
#define PNGARG(args)            args = 0

and then the png proto is defining our fp for us. (unfortunately png.h sticks "extern" in front of all the protos, so you have to copy out the protos and take off the extern).

So anyhoo there you go, libpng use with delay-load.

BTW this also suggests a way that you can make your DLL very easy to use with delay-load and manual imports. What you should do is provide a header with only your function protos in it, and wrap them with a macro like :


DECLFUNC( ret, name , args );

eg.

DECLFUNC( png_voidp, png_get_io_ptr, (png_structp png_ptr) );

Then a client could just include that header multiple times and change DECLFUNC to various things. For example if you had a header like that, you can look up all the func names at LoadLibrary time, instead of doing each one on first use (this removes a branch from each function call site). eg :

#define DECLFUNC( ret, name , args )    ret (CALLBACK * STRING_JOIN(fp_,name)) args = 0
#include "allfuncs.inc"
#undef DECLFUNC

void ImportAllFuncs()
{
HMODULE hm = LoadLibrary;
#define DECLFUNC( ret, name , args )    STRING_JOIN(fp_,name) = ImportFunc(name,hm)
#include "allfuncs.inc"
#undef DECLFUNC
}

Unfortunately you can't do a #define inside a #define or this could be used to alias the names transparently with something like


#define DECLFUNC(ret,name,args) #define name (* STRING_JOIN(fp_,name) )
#include "allfuncs.inc"
#undef DECLFUNC

(ADDENDUM : or, you know, just use the /DELAYLOAD option)

11 comments:

Mārtiņš Možeiko said...

If you are using Visual Studio, then it can do all this automatically. Check out "Project Settings -> Linker -> Input -> Delay Loaded Dlls" setting. Then dll file will be referenced from application, but it won't be loaded until somebody actuall tries to call function from it.

cbloom said...

Holy crap. LOL.

Carsten Orthbandt said...

You can get libpng to support static linkage. Let me know if you want the gory details.

Anonymous said...

I used to do the #include "foo.inc" for lists of things and include it multiple times, but I've swung around to instead doing

#define ITEMS item(whatever1) item(whatever2) item(whatever3) ...

With each item on one line and backslashes between them. This seems a bit less aggressively obfuscated.

cbloom said...

Oh yeah, I actually switched to that method myself a while ago, but then forgot about it and switched back to the multi-include. You're right that it feels cleaner somehow.

cbloom said...

@Carsten - sure, share the knowledge; you can email me or post it.

I do hope that everyone realizes by now that DLL's are a total disaster and you should statically link everything you can. Dlls are an annoyance for the client and a failure mode for your app. Remove failure modes whenever possible. Robustinate!

Fabian 'ryg' Giesen said...

Interestingly, the main way a lot of people use DLLs is not to factor out common code (as is the intention), but merely to split compilation into several chunks that each have acceptable link times.

Once you start thinking about this, it immediately becomes clear how unbelievably broken this is - linking is getting too slow, so now we do it every time the process is launched?

Of course the key is that a) there's no dead stripping across module (app/DLL) boundaries, and b) DLLs only contain symbols that you intentionally export, not tons of autogenerated name mangled gunk that happens to be publicly visible even though there's no good reason for it.

I'd really like a notion of "packages" with DLL characteristics (all internal dependencies resolved, LTCG "island", own scope, only explicitly marked symbols visible from the outside), but still used for static not dynamic linking. Of course that requires innovation in linkers, something that doesn't seem to have happened in over 20 years.

cbloom said...

@Unknown - totally.

It's one of those things for my list of "what the C standards committee would do if they were actually programmers".

Perhaps the #1 problem with C is that sharing work between disparate projects is a total disaster.

Packages would be at least a step in the right direction. It would isolate things like "hey I assume a certain override for operator new" which are part of what make sharing C such a clusterfuck.

Carsten Orthbandt said...

We don't use a stand-alone static lib of libpng. Instead we included the required source code in our core lib.

The files you actually need are:
png.c/.h
pngconf.h
pngerror.c
pngget.c
pngmem.c
pngpread.c
pngpriv.h
pngrio.c
pngrtran.c
pngrutil.c
pngset.c
pngtrans.c
pngwio.c
pngwrite.c
pngwtran.c
pngwutil.c

We removed the zLib from libpng and used the one we have already. That requires a change of the zlib include path on png.h (line 397 im my version).

We also changed all the PNG_EXPORT function decls to "extern" to make C++ happy.

And we had to disable PCH on the libpng .c files of course.

That's pretty much it.

Carsten Orthbandt said...

Oh, and I wholeheartedly agree on the DLL rant. Major PITA with no gain whatsoever.

ryg said...

Or you could, you know, use stb_image. (That "Unknown" above was me by the way, apparently Blogger doesn't understand Google's multiple sign-on stuff that I need to do to access both my private and work GMail account. Sigh.)

old rants