10/12/2008

10-12-08 - 2

What actually happens when you have a big static array in your executable? Like
static char c_bigBuff[65536] = { 0 };
I believe that executables have a special "zeroed statics" section, and when the exe loader does its thing, it grabs pages pages for them and just points them at the zeroed pages. I remember on the XBox I was worried about our static initializers taking up space in the executable, and it turns out *zeroed* static values do not take up any space, but non-zero ones do obviously.

I do know that Windows has a page zeroer thread which is always running. It's at the lowest possible priority and all other threads are higher. It runs in the idle and just zeros free pages. I don't know if these are actually used to speed up page zeroing, like when I ask for a zeroed page do it give me a pre-zeroed one if possible and not burn any time doing the memset?

Anyhoo, somebody school me.

Okay, yeah, this stuff goes in the "BSS" in most executable formats. BSS is sometimes called "uninitialized" data though of course it's really zero-initiliazed, but this comes from an evil fucking old C-ism :

Uninitialized statics are gauranteed to be zeroed in C !!


void func()
{
	int x;
	static int y;

	... code ...
}

Here "x" is undefined (maybe a bug), but "y" is defined by the standard to be zero. Yay C. That's evil.

Apparently under Linux, when you load your exe the BSS points at a virtual COW "zero page". If you only ever read those zeros it never copies.

Whoah, Won just blew my mind. Obviously all pages that the OS gives to an app are always zeroed! This is for "security" just like how sectors of disk that you get access to by extending a file are zeroed. When the executable is loaded, all the pages that it consumes are zeroed before it does anything. The way the static data is loaded apparently is just that the data with non-zero initializers is put at the front of the section, so the exe just asks for a big block, and then values are only copied into the front, the rest stays zero. So if you have a big static array, it's getting zeroed whether you ask for it or not.

So the question then is how does VirtualAlloc zero memory, and I assume the answer is that it tries to use pages already zeroed by the zero-thread, but if there aren't any available it memsets.

Okay yeah, I've had that confirmed by hearsay anyway. In fact, free pages in windows go into a dirty pool, the background zeroer thread takes pages from dirty and puts them in a zero pool, new allocs try to take pages from the zero pool. There's this really sweet looking app : MemInfo from Alex Ionescu , but it only runs on Vista. Actually Vista has a lot of sweet new APIs that let you do some new low level things. Anyway it shows you info about how many pages are in each type of pool.

No comments:

old rants