11/08/2011

11-08-11 - Differences Running in Debugger

Bugs that won't repro under debugging are the worst. I'm not talking about "debug" vs "release" builds; I mean the exact same exe, run in the debugger vs. not in the debugger.

What I'd like is to assemble a list of the differences between running under the debugger and not under the debugger. I don't really know the answers to this so this is a community participation post. eg. you fill in the blanks.

Differences in running under the debugger :

1. Timing. A common problem now with heavily threaded apps are bugs due to timing variation. But where do the timing differences come from exactly?

1.a. OutputDebugString. Duh, affects timing massively. Similarly anything you do dependent on IsDebuggerPresent().

1.b. VC-generated messages about thread creation etc. These obviously affect timing. You can disable them being shown by right-clicking in the output window of the debugger, but the notification is still being sent so you can never completely eliminate the timing difference for creating/destroying threads. (and the debugger does a lot more work for thread accounting anyway, so create/destroy will always have significant timing variation).

2. Exceptions. (not C++ exceptions, which are handled pretty uniformly, but more the low level SEH exceptions like access violations and such). Obviously in the debugger you can toggle the handling of various exceptions and that can change behavior. One thing I'm not sure of is if there are any registry settings or other variables that control exception behavior in NON-debugged runs? (* more on this in another post)

3. Stack. Is there a difference here? Not that I know of.

4. Debug Heap. This is probably the biggest one. Processes run in the debugger on windows *always* get the debug heap, even if you didn't ask for it. You can turn this off by setting _NO_DEBUG_HEAP as an environment variable or starting MSVC with -hd. See Behavior of Spawned Processes .

Note that this isn't coming from MSVC, it's actually in ntdll. When you create your process heap, ntdll does a "QueryInformationProcess" and sees if it's being debugged, and if so it stuffs in the debug heap. The important thing is that this is at heap creation time, which leads to a solution.

5. Child Process issues. Because the debugged process is a child process of the debugger, it inherits its process properties. (the same issue can occur for running under "cmd" vs. spawning from explorer). Two specifics are "permissions" and environment variables. Another inherited value is the "ErrorMode" as in "GetErrorMode/SetErrorMode".

There's a solution to #4 and #5 which is this :

Start your app outside of the debugger. Make it do an int 3 so it pauses. Then attach the debugger. You can now debug bug you don't get some of the ran-from-debugger differences.

(note to self about attaching : for some reason the MSVC "attach to running process" seems to fail a lot; there are other ways to do it though, when you get an int 3 message box popup you can click "debug" there, or from task manager or procexp you can find the task and click "debug" there).

6 comments:

ryg said...

Attaching: I like "while (!IsDebuggerPresent()) Sleep(500);" for this - works even if the Just-in-Time debugging stuff is flaky (as it often is).

Tom Forsyth said...

Don't know if the runtime leak/scribble checking is at compile time or debugger time. But it adds padding to stack & allocation and changes page tables. Indirect effects are of course that it changes alignment, etc.

cbloom said...

Tom - yeah I was thinking about that stuff, but so far as I can tell it's not affected by being in debugger or not (as long as you don't actually trigger one of the checks)

Jonathan said...

I actually find the bit of code on this page useful for causing an attach.

http://www.codeproject.com/Tips/254524/Self-debugger-attach-to-process?display=Print

Because it calls the jit debugger directly it doesn't rely on the OS bringing it up which does seem not to work sometimes.

I skip the last bit that actually stops the process.

Dark Helmet said...

http://stackoverflow.com/questions/6103314/visual-studio-debug-iterators

Just flipping between debug and release can render your binary app incompatible with libs it uses. Annoying...

Philip Taylor said...

There's a paper "Producing Wrong Data Without Doing Anything Obviously Wrong!" showing how factors like the number of bytes of environment variables can alter the stack layout on Unix and significantly affect performance. Don't know if that particular case applies to Windows too.

old rants