Debugging DLLs

[Digby]

Well this is just great.  I've discovered that the eVC debugger won't break inside a DLL's entry function (DllMain).  Since the OS is directly calling DllMain the debugger thinks the OS is inside a system call and won't allow breakpoints or debugging.  I even tried throwing in a DebugBreak and it ends up causing the debugger to hang.<br><br>I ran into something similar to this when I specified a callback function for the audio subsystem to notifiy my app using the waveOut APIs.  My code in the callback was getting executed but I could never get the debugger to hit a breakpoint inside the callback.<br><br>Anyway, just wanted to give you guys a heads-up in case you run into this.  I suppose I don't have too much room to complain since the tools were free.<br><br>

[Boogaloo_Shrimp]

I don't think i've played that level on starcraft yet.

[Digby]

Sometimes I wonder why I even bother with this website.  This place is more like a teen IRC channel than a development community.  My patience is wearing thin.<br><br>

[RICoder]

Digby, did you try doing an int13 call?<br><br>Shit, you can't do inline asm in eVC++ can you...crap.<br><br>Thanks for the heads up.  I have a question for you now.  Is it only in the DllMain function or is it in ALL Dll functions?

[Digby]

DebugBreak forces a trap on the ARM (int 3 equivalent on x86).  eVC doesn't support inline assembly, so you would have trouble using this method anyway.<br><br>It's only in DllMain because the OS is directly calling DllMain, rather than from your application.  Other functions in the Dll are called from your app so the debugger will work with them.<br><br><br><br>

[RICoder]

ok.  I see.  hmph.<br><br>Doesn't the OS always invoke the DllMain function?  Even in x86 land?  I thought it did, so I am not sure why this would be the reason why it wouldn't trap for you.

[Digby]

The reason is that the debugger is broken in eVC.  <br><br>The eVC debugger is a "user mode" debuggger.  That means when it breaks, only a single process is interrupted rather than the entire OS (kernel mode debugger).  Apparently, the debugger thinks that DllMain and various callback functions are part of the OS and not part of a separate user mode process.<br><br>

[RICoder]

Right right right...ok, I get it.<br><br>Thanks.<br><br>How did your question turn into a Q&A for me?