4/29/2007

04-29-07 - 1

"Edi" is a really great movie. I'm really sick. God I hate being sick.

4/27/2007

04-27-07 - 1

Well, I just learned a valuable lesson. The fire alarm panel should not get wet. Dear god my ears.

4/26/2007

04-26-07 - 1

Found a cute little summary document : Windows NT Kernel Overview (PDF) with lots of internal goodies. Quite interesting.

4/25/2007

04-25-07 - 2

I'm so not a Firefox configuring geek, but this is cool : Change your FireFox default URL action ; I like to just have it do "I'm feeling lucky" so I can semi-type a url and it works.

Also I got this problem where clicking an HTML file would open up Firefox twice. If you go to "File Types" and look at the HTML config it had put "Firefox %1" as the command and also this whole DDE section. I just turned off DDE and it works fine. Probably taking out the %1 and leaving in DDE would work also.

04-25-07 - 1

So I'm playing poker again, 200NL on PokerStars. I'm still a little drained and bored of it, but I need some money and I don't want to work. I can make around $100/hour there and more if I can move up. I'm going to try to really make another run at getting better and moving up. Of course the games are way tougher now than they used to be, but I have some aces up my sleeves ;)

Here's one of my more exciting hands so far, and also a demo of my sexy html hand output :


No Limit Hold'em Ring Game (6 max) , 5 players
Blinds : $1/$2

Stacks:
UTG : $217.05
CO : $255 (Hero)
Button : $200
SB : $201.65
BB : $92.45

Pre-flop: (5 players) Hero is CO with Qh Jh
UTG folds, Hero raises to $8, Button raises to $24, SB calls ($23), BB folds, Hero raises to $88, Button folds, SB calls ($64).

Flop: 8s 8d Js ($202, 2 players)
SB checks, Hero bets $152, SB calls all-in.

Turn: 5h ($467.65, 2 players)
River: 2d ($467.65, 2 players)
Results: (final pot $467.65)
Hero uncalled bet of $38.35 returned
Hero wins $465.65 (+$225.65) with Qh Jh (Two Pair)
SB mucks 4d 4h (Two Pair)

Button here is an aggro TAG so he's raising me light a lot. SB is a horrible donk so I'm not worried about his cold-call of a reraise (which would normally be scary) - but I can use him to squeeze preflop and I know Button *is* worried about SB so I can get him out. I was a bit surprised SB called down so light, but that's just a bonus ;)

4/20/2007

04-20-07 - 1

I can totally sympathize with the idea of going nuts and shooting people, but shooting a bunch of graduate students and professors is just insane. If you're going to rage at the world and take yourself out, take out some politicians, some CEO's, some lobbyists, some police, some models, some yuppies, some televangelists, something sensible!

4/19/2007

04-19-07 - 1

If I was in elementary school and I saw a kid with one of those flight-attendant pull-behind suitcase/backpack things that all the yuppie kids have now - I would definitely beat them up.

4/16/2007

04-16-07 - 1

I've always wanted to make my apps clean Console/Windows apps. What I mean by that is - if I'm started from a console, I can either attach to that console or not, while if I'm started from an icon (no parent console) I can either pop up my own console or not. Turns out this is impossible because of the ass way Windows handles consoles and spawning processes. There is this nice function AttachConsole() which it seems like it would do the trick but basically it seems totally useless.

The goal is to have an app that is primarily a GUI app, but it has processing and help modes where you might want to just run it on the command line. For example, if someone runs "cbapp -?" and they do it from a cmd, I want to just output the help in the CLI, I hate it when apps pop up their usage in a little MessageBox. A classic example is the "msdev" or "devenv" app, which by default pops up the MS Visual Studio GUI interface, but you can run it from a CLI to do non-GUI builds of your projects and things like that. Basically I think pretty much every single app should be designed this way.

Anyway, there is a solution, but it's a bit ugly. First of all, you compile your whole app twice, once for CONSOLE and once for WINDOWS subsystems. The two apps are in the same directory and have the same name, but the console one is named ".com" ; because of the cmd execution order rules if you run the app from the command like you will run the .com version, obviously shortcuts point at the .exe version. Now, the .com version will attach to the console - if you decide it's in a mode where it should be a detached windows app you have it spawn the .exe version. Conversely, if the .exe is run and you decide you need a console, you can just call :

if ( AllocConsole() ) // does nothing & returns false if we already have one
{
	freopen("CONIN$","rb",stdin);   // reopen stdin handle as console window input
	freopen("CONOUT$","wb",stdout);  // reopen stout handle as console window output
	freopen("CONOUT$","wb",stderr); // reopen stderr handle as console window output
}

Which puts a console on your Windows app. Apparently several MS apps use this .com/.exe trick.

Now, there is one last problem that I hate. The console window you pop has a close box on it, and this close box has a nasty property - when it's clicked your app is *terminated* like a KillProcess() - you don't run any shutdown code! This is pretty evil and retarded IMO. I haven't found my ideal solution to this but the two things that come to mind are : 1) hook/hammer that window to get rid of its close box, or 2) use one of the many Windows console replacement libraries (eg. don't use the built-in windows console at all). You could also write your own console pretty easily, especially if you're only doing output. You just direct stdout to a windows file and wait on it in a thread to update a little text pane window.

Part of the weirdness with the console window is that even if your app makes the console you don't own it. All console windows in Windows are owned by "csrss.exe" which an always-running process which acts as the console server. When you call AllocConsole() it actually sends a message to csrss and tells it to make the console window; it also makes some memory mapped files and gives you the handles to them which are then your stdin/stdout/stderr. To talk to the console you write those files, it sees they changed and update the window. This is what makes the console totally async from your app and means you app can crash and still have a valid console, which is what makes it nice for debugging. The csrss thing sort of explains why you just get a KillProcess. When the close box on the console is hit, csrss gets that message, sees the PID that's attached to that console and just kills it.

Addendum :

Actually it can be easier than this. You don't need to do the whole .com/.exe thing unless you want your app to be significantly different in windows & console mode. Instead you can just do this easy thing :

Make your app a console app with a normal main() entry point. When your app decides that it wants to detach and be a windows app without a console, you just respawn yourself like this :

	// I'm consoled and I don't want to be
	fprintf(stderr,"Respawning detached\n");

	STARTUPINFO si = { 0 };
	PROCESS_INFORMATION procinfo = { 0 };
	si.cb = sizeof(si);

	BOOL ok = CreateProcess(
		NULL,
		GetCommandLine(),
		NULL,
		NULL,
		FALSE, // 	bInheritHandles
		CREATE_NO_WINDOW | DETACHED_PROCESS,
		NULL, // = inherit environment
		NULL, // = inherit curdir
		&si,
		&procinfo);

	if ( ! ok )
	{
		fprintf(stderr,"ERROR : CreateProcess failed !\n");
	}

	exit(0);

This will relaunch your app separated from the parent handles (so if you're run from cmd you don't inherit its stdin/stdout). Your app is still a console app, it just won't have a console and the stdin/stdout will be null files. You can always pop your own console in the future too.

Actually there is one ass crappy thing about this - if you run this from a shortcut it first starts as a console app, a console will pop up, then you'll respawn and the console dies, so you have this evil flash at startup. So, you can solve that by going the .com/.exe route, and I still don't have a happy solution to this problem. If you do the proper thing and use the .com/.exe trick then your .com just spawns the .exe using the above code, (but you have to change the app name in the commandline).

Another semi-related tip you might not know : there's really no point in defining _CONSOLE or whatever in your project, and in the linker you can set subsystem=NOT SET. Then MSVC will just autodetect whether you implemented main() or WinMain() and do the right thing.

More addendum : Tom points out you can fix the console-making-your-app-terminate thing. You just implement a ConsoleControlHandler and catch CTRL_CLOSE_EVENT. I have no idea why Windoze didn't make the default version of that raise a signal, which would've made all the ANSI apps work nice, but they didn't. The only funny bit is that a ConsoleControlHandler is run on a little thread that's created in your app just to run it (presumably with CreateRemoteThread from the csrss). That means you can't just PostQuitMessage() because you're not on the right thread, and you also can't just PostMessage() to your main window unless you're a bit careful (okay, in practice you probably can do that 99% of the time). Anyway, a safe thing to do is like this :


volatile LONG g_consoleClose = 0;

BOOL MyConsoleCtrlHandler(DWORD fdwCtrlType) 
{
	// note : this is run on a little thread inside my process which is created just to run this
	//	that allows me to pop a message box or something if I want
    switch (fdwCtrlType) 
    { 
        // Handle the CTRL+C signal. 
        // CTRL+CLOSE: confirm that the user wants to exit. 
        case CTRL_C_EVENT: 
        case CTRL_CLOSE_EVENT: 

			InterlockedIncrement(&g_consoleClose);
 
            return TRUE; 
 
        default: 
            return FALSE; 
    } 
} 

And then in your main thread you do this :


		if ( InterlockedExchange(&g_consoleClose,0) > 0 )
		{
			PostMessage(g_mainWin.GetHwnd(),WM_CLOSE,0,0);
			break;
		}		

Or whatever you want to do in there to signal your app its time to get out.

4/14/2007

04-14-07 - 1

I wrote a while ago that Layered Windows under MS Windoze are very slow. Well, like a retard I didn't actually do the profiling and it turns out I'm totally wrong. Layered Windows are in fact nice and speedy, and you can have at least one full-screen alpha-blended windows at 60 fps. What *IS* slow is pretty much any GDI function call, in particular GetPixel & SetPixel are just insanely slow. Fortunately it's really easy to just get your DIB bits as a DWORD RGBA array and do all your own drawing on that in-memory bitmap, then you just UpdateLayeredWindow and speed away.

4/13/2007

04-13-07 - 1

Carnitas = pork confit. Now you can charge $20 for it instead of $1.

4/12/2007

04-12-07 - 1

IC tipped me off to Lame Blog which is a plain text -> blog thing just like I want, but I haven't figure out how to get it working yet. Someday you will be able to comment! In the mean time I retardedly report interesting comments :

Bernanahan pointed out the Reflection() thing may break down in large code bases simply because it does force you to put a lot of junk in the header which = lots of recompiling. Yeah, that's the biggest problem with all template magic junk, it might be crippling on game engines. My apps still take just a few seconds to compile so I'm not sweating. Also if you're really gonna use Reflection to define user-editable values you probably want ranges & descriptions and lots of other metadata. Of course you can do this in the Reflection system but it starts to get ugly and the pros & cons swing more towards a custom markup & parser system.

Sholz wrote about my security ramblings. I'm a total newb on this stuff, I just figured out how to hook & scrape to get the info I need. Anyway, he's more in on the whole rootkit and anti-rootkit crowd which is a big field I guess. All the anti-rootkit people basically know that Windoze is hopelessly insecure, so the thing they do is scan for hooks & patches, and/or take measure to prevent them from getting installed. So then it becomes a war of getting around each others' blocks & hiding from each other. It's quite impossible to make a Windoze box secure - all you can do is make it secure from current known attacks, and make it difficult to write new attacks that aren't blocked/detected. (this is assuming the attack is invoked by running a user process - it's possible in theory to block all remote attacks if all your interface software is written well and bug free).

4/11/2007

04-11-07 - 3

Random thoughts on how you might actually make an app safe from snooping in Windows :

First of all, you can't call any OS functions with data that you want protected. It's very easy to hook any OS calls. Now, lots of things in the C std libs go to OS calls - even things like strcpy (!!) - so if you want to write a program in C you probably need to write your own "safe CRT" that doesn't call to the OS. Now, you can of course still call to the OS (you have to) but you should only call it with data that's protected, for example when you send network packets they should be already encrypted.

It's possible that any happen which uses HWND's is unsafe. It's easy too hook HWNDs and manually send various messages and such. I'm not sure about this but if you really want to be safe you might have to do all of your own windowing. You could do this in an OS-friendly way by making a separate app which just maintains a blank window with a title bar. That way people can still alt-tab and z-order this blank window, and then your real secure app which is in its own process just gets the location of that window and draws in it manually.

Now, any drawing with GDI can be hooked & screen capped, so that's not safe. You might just be in trouble here because pretty much any drawing is going to require an OS call which can be hooked. There are things you can do, such as YUV overlays which don't go through the normal GDI and can't be screen-capped, however to fill them you would still have to pass in the bitmap to draw and someone could hook that call to grab your bits.

One thing you might be able to do is detect hooking. I'm not sure how you could do this but conceivably instead of just calling to the OS you could GetProcAddress on the calls, and then look at the code you could be calling and scan for jumps in it to see if it's a thunk out to non-kernel space, then just don't make the call.

Of course your memory is still exposed. Memory in windows is totally unsafe and other apps can even get your list of virtual memory pages AND change the protection on them. You might think you could make yourself safe by taking all your pages and setting them to NOACCESS , but anyone can grab the page and change it, so this is pretty retarded.

Other apps can also CreateRemoteThread on you to insert code into your process and I'm not sure if there's any way to forbid that.

The easiest way to get code to run in another process is with a windows hook, but I think it is possible to write an app that simply doesn't cause any hookable events.

04-11-07 - 2

Critical Sections don't work across processes (mainly just because they do local allocs, so they have internal handles and pointers which are process-specific). (I guess there's also some differences between windows process-switch and thread-switch logic which would call critical sections to be unsafe across processes even if their memory was global).

Windows doesn't have any drop-in cross-process data locking thing. Fortunately some MS dude wrote a cool bit of free code : Metered Sections

There is one bad thing about the Metered Section code as is, which is that the Open and Close use malloc and free which is not safe to call inside DllMain. My preferred fix to that is to expose the in-place Init() and Delete() analogous to Critical Sections, so that you can just have a Metered Section structure in a static variable and initialize it in place with no allocs. If you wanted them to be safe & dynamic you'd have to use TlsAlloc or something.

04-11-07 - 1

I did this Reflection + Prefs thing a while ago and I'm using it now and it's just so sweet I thought I'd write about it a bit.

It's in the Galaxy3 code which is here . (that version is old and my new stuff is better, I'll put it up some day)

There are a few components so I'll go through it. This is a very C++ template metaprogramming thing. When I wrote it I was constrained by the limited features of VC6, and it works in VC6, but if you use the fully compliant spec you can do even nicer things.

"Reflection". This is just a convention. A lot of template metaprogramming is just conventions. Anyway, this convention says : any reflectable class defines a member function called "Reflection" which reflects all its members. The Reflection function takes a templated visitor so it can be visited by any functor, and it just calls that visitor on all its members (with their name).

So, having this convention is cool already, because if you implement Reflection in all your classes you can now do an "IOZ" type of automatic serialization system and you can serialize any class which implements Reflection.

In this case we're going to use Reflection to do "Pref" IO. Prefs in my world are text files like INI files, that have a bunch of lines something like "member : data" in nice readable and editable formats.

To read & write prefs we need to know how to read & write various data types to text. To do that, we define two templated functions : ReadFromText and WriteToText. You then specialize them to specific types and write the text IO for those types. For example to do ints you would supply functions that basically do printf and scanf. If you want you can implement these for your own arbitrary types, but you don't do that very often, only for new low-level types.

The default implementation of ReadFromText and WriteToText assume the unknown type is a complex type, so they write it in braces in the text file, and then call "PrefIO()" on that type so it can IO itself (in text).

PrefIO is just another convention. There are no implementations of PrefIO for basic types - if your type was basic it would have been IO'd by ReadFromText/WriteToText. The default implementation of PrefIO just calls Reflection !! Okay, now this is where the magic is starting to happen. Default PrefIO assumes the type is a complex type that implements Reflection, so we just pass in the IOText functor which is automatically called on all the members. (another case where you might want to implement the PrefIO function is if your type is an opaque class with a pImpl - you don't want to expose Reflection publicly so you just expose a PrefIO function which can then call through to work on the hidden pImpl).

Now, there are some types which are neither basic (and get handled by RFT/WTT) , nor are their Reflected. One example is any complex type that you don't own the code to, so you simply can't add a Reflection() member. For example, types provided by another library such as Windows. You still want to do PrefIO on those types, well you can - you just specialize the PrefIO template for those types and manually call the members instead of having Reflection do it for you.

Okay, so what's so cool about this system? Well, for doing basic stuff it's not particularly better or worse than any other reflection/member-IO system. In any system like that you have to mark up a list of your members somewhere and we have to do that too. (I'm ignoring the fact that you can autogenerate that list by parsing headers, since you can do that in any system, and you still have to mark it up by flagging which members to reflect and which not to).

The cool thing is what happens when you encounter a type that the system isn't familiar with. In traditional prefs text IO systems, first of all that error might not be detected until run-time, which is nasty. Second of all, to support IO for a new type you often have to define some new MetaData object that describes that type, and add it into the Pref system.

In this system when you add some new type to a class, you will have a Reflection() in that class that acts on the new type. When you write code that treats that class as a pref, it will try to compile the PrefIO functions, which will produce a compile error that looks something like :


c:\src\cblib\Reflection.h(211) : error C2228: left of '.Reflection' must have class/struct/union type
        type is 'COLORREF'

You're getting this error because it's falling down to the backup implementation for unknown types, which is to assume they're complex and call Reflection() on them. Since this type doesn't have Reflection you get an error. Now you can fix that either by -

1. if it is your class, you just implement Reflection()

2. if it's a basic type that you just don't have an IO for you, you write ReadFromText/WriteToText. If these are found they are used first and it doesn't even call to PrefIO.

3. else it's some sort of struct or type you don't own, (like in this example), so you write a PrefIO, which overrides the default so Reflection doesn't get called.

Okay, the other really great thing about this is what happens any time it breaks for whatever reason. You have some weirdo ill-behaved class and the standard generic Reflection doesn't work. Well, your Reflection() function is just code and there's no reason you can't just go in there and write extra code to fix whatever is wrong. In the most severe case you could specialize Reflection to specific visitors to special-case those, but more often you can fix it by just doing some time addition to normal reflection.

IMHO there's also a big cognitive benefit to having a system where the compiler is doing the book-keeping for you (errors are compile time and the compiler tells you if you don't support certain types), and where the system is "just code" as opposed to some custom metadata or parsing system which will have its own quirks and ways of working that you'll have to learn.

4/10/2007

04-10-07 - 1

This blog has a cool article on how Win16 Hooks were implemented. Some very cute stuff.

4/07/2007

04-07-07 - 2

Here's a nasty bug to avoid : DLL Shared data only works if it's *initialized*. Uninitialized data will just silently not be shared. So evil. So like this works :


#pragma data_seg(".SHARED")

char buffer[80] = { 0 };

#pragma data_seg()
#pragma comment(linker, "/section:.SHARED,rws")

And this will appear to work just fine but not actually be shared :


#pragma data_seg(".SHARED")

char buffer[80];

#pragma data_seg()
#pragma comment(linker, "/section:.SHARED,rws")

One thing I'm actually not sure about is whether you can put C++ classes with constructors in the DLL shared data space and have that work right (eg. be shared, and just be constructed/destructed once when the last DLL instance dies).

04-07-07 - 1

So, I might get back into poker. Just sort of for laughs & the challenge I've been playing with scraping data from certain poker sites that try to make it difficult to scrape data. LOL. Windows has the retarded "make it difficult" security system where any user-level app can completely access any other user-level app, all they can do is make it sort of difficult. In the end you can always go and grab their virtual memory information and walk their memory and do whatever you want. You can also insert yourself as a DLL and hook their function calls to the OS and easily grab info and change behavior that way. ALL YOUR BASE ARE BELONG TO US!! It's pretty fun, I've never done this stuff before and it makes me feel like a real l337 h4x0r. Windows is ridiculously unsafe.

For various reasons I don't want to give away too many details, but I will say that the only really quality information comes from MS themselves, they have a lot of articles on how to write debuggers and API tracers, which is of course also how to write great hacks. All of the script-kiddie web sites are useless so far as I can tell.

4/02/2007

04-02-07 - 2

You may not be aware of it at all, but there are currently thousands of huge private equity hedge funds doing computerized trading on all kinds of markets (stocks, currencies, commodities, futures, etc.), moving billions of dollars. They generally do risk-free or very low risk transactions. These aren't traditional investment funds that put money into businesses they think will succeed. These funds look for "loops" or ways to "bet both sides" , like arbitrage or just plain mistakes. They also take advantage of the different schedules of the markets and the delays of big mutual funds.

What is the real net result of these funds? The funds make a lot of money, and everyone else makes less. The money these guys are making doesn't come out of the air. Then they trade a commodity vs. a future, it's because they spotted a price that was incorrectly too low, and by trading it they drive it up and make it correct, and suck the "slack" out of the system. That sounds harmless, but the slack was a discount for someone buying that commodity which made the goods cheaper. The same thing occurs in stocks - effectively they are "fixing" the prices of various transactions, which always means making them worse for others.

This has been going on for perhaps 10 years, but has really only kicked into high gear in the last 5 years. We won't know the full effects on long term market returns for many years, but already something measurable has happened. Mutual funds that track the major indexes (like the S&P 500) are performing much worse than they used to, perhaps 1% worse. This is happening because changes to the S&P inded are announced in advance, and it takes a while for the mutual funds to shift in, and the hedge funds sit in the middle and make profit.

04-02-07 - 1

I'm reading through "A People's History of the US" and will post some of the interesting things here. It's definitely a left-wing commie manifesto, but pretty interesting so far, worth a read.

He notes "a contract among unequals is usually written as if the parties were equal, but with varying power to enforce the contract". This really struck me as a tool that is constantly used today and is very powerful. The contract looks good, it makes you think you have all these rights, but in reality the powerful player can enforce their side of the contract, and you have little/no power to enforce your side. Most of us peons are in these contracts all the time - with the gas company, with our employer, with our health insurance, with the government. In theory we have all these rights but in practice they can pretty much do anything to us and we have no recourse unless the sin is grevious enough to sue and get a conviction. As long as their careful to only fuck us over in minor ways, they win. For example, they can constantly send us contract revisions that we really have no choice but to accept.

4/01/2007

04-01-07 - 1

Wines - why not more blends? There are so many single grape wines in America that are very flawed, and blending them could easily fix them. You might have a Zin that's too dry and peppery, a cab that's too big and tannic with no other notes - just pour em together and you have a much better wine!! It seems there's a single-grape snobbery in America that definitely doesn't exist in France (pretty much all their top wines are blends). Some of the ideal blends might take wines from different wineries which creates some logistical difficulty, but there are plenty of big producers that could easily make superior wines.

old rants