Choosing the Debugger at Crash Time
It's great to have JIT debugging bring up the debugger when your application crashes, but the big limitation is that you can have only a single debugger at a time in the Debugger value. As we'll see in later chapters, the various debuggers you have at your disposal have strengths and weaknesses depending on particular debugging situations. Nothing is worse than having the wrong debugger pop up when you really want the one you know will simplify solving that bug you've spent weeks trying to duplicate.
This was a problem begging to be solved, so I set out to do so. However, because of what look like bugs in the way JIT debugging starts under Visual Studio .NET, I had to work through a lot of trial and error to get my approach working. Before I can discuss the problems, I need to show you how Debugger Chooser, or DBGCHOOSER for short, works.
The idea behind DBGCHOOSER is that it acts as a shim program that gets called when the debuggee crashes and passes on to the real debugger the information necessary to debug the application. To set up DBGCHOOSER, first copy it to a directory on your computer where it won't be accidentally deleted. The operating system tries to start the debugger executable specified in the Debugger value of the AeDebug key, so if the debugger isn't available, you won't get any chance to debug the crash. To initialize DBGCHOOSER, simply run it. Figure 4-1 shows the configuration dialog box in which you set the appropriate commands for each of the various debuggers supported by DBGCHOOSER. The initial run of DBGCHOOSER is set with the appropriate defaults for most developer machines. If you don't have the various debuggers in the path, you can specify the appropriate paths. Pay special attention to the Visual Studio .NET debugger path because the JIT shell used by Visual Studio .NET isn't in the path by default. After you click OK in the configuration dialog box, DBGCHOOSER writes your debugger setting to an INI file stored in your Windows directory and sets itself as the default debugger in the AeDebug key of the registry.
w Configure Debugger Chooser PT
Visual Studio.NET debugger command line: J Files\Microsoft SharedWS7Debug\vs7jit"exe'' -p Z\d -e Z\d Visual C++ 6 debugger command line: jrnsdev.exe -p %\d -e %ld WinDBG debugger command line: jwindbg -p~%id -e %ld Dr. Watson debugger command line: drwtsn32 -p Z\d -e Z\d -g
■ The "-p Zld -e%ld" must always stay in that order.
- Pressing OK sets DbgChooser as the default debugger
- Either specify enact paths to the debuggers or ensure they are in the path.
- Make sure to restart TASKMGR after pressing OK.
Cancel
Help
Figure 4-1: The DBGCHOOSER configuration dialog box
After you experience what I hope is one of your rare crashes, you'll see the actual chooser dialog box pop up when you click Debug in the crash dialog boxes, as shown in Figure 4-2. Simply select the debugger you want to use and start debugging.
debugger Chooser
'isual Studio.NET
Visual C++ 6.0 Debugger
WinDBG
'r Watson
Do Not Debug This Crash
Figure 4-2: The DBGCHOOSER debugger chooser dialog box
The implementation of DBGCHOOSER is nothing exciting. The first point of interest is that when I called CreateProcess on the debugger the user chose, I had to ensure that I set the inherit handles flag to true. To ensure everything is copacetic with the handles, I have DBGCHOOSER wait on the spawned debugger until it ends. That way I know any necessary inherited handles are still there for the debugger. Although it was more of a pain to figure out than interesting to implement, getting Visual Studio .NET to work properly from DBGCHOOSER took some doing. Everything worked like a champ with WinDBG, Microsoft Visual C++ 6, and Dr. Watson. However, when I'd start Visual Studio .NET (actually VS7JIT.EXE, which in turn spawns the Visual Studio .NET debugger), a message box would pop up indicating that JIT debugging was disabled and wouldn't start debugging.
At first I was a little stumped about what was going on, but a quick check with the wonderful registry monitor program (Regmon), from Mark Russinovich and Bryce Cogswell at www.sysinternals.com, showed me that VS7JIT.EXE was checking the AeDebug key's Debugger value to see whether it was set as the JIT debugger. If it wasn't, up popped the JIT debugging disabled message box. I was able to verify that this was the case by stopping DBGCHOOSER in the debugger while DBGCHOOSER was active because of a crash, and changing the Debugger key back so that it pointed to VS7JIT.EXE. I have no idea why VS7JIT.EXE feels it's so important that it won't debug unless the debugger is the JIT debugger. I did a little quick coding in DBGCHOOSER to fake out VS7JIT.EXE by changing the Debugger value to VS7JIT.EXE before I spawned it, and all was good with the world. To get ® DBGCHOOSER.EXE reset as the JIT debugger, I spawned a thread that waits for 5 seconds and resets the Debugger value.
As I mentioned when I first started talking about DBGCHOOSER, my solution isn't perfect because of problems with JIT debugging in Visual Studio .NET. On Windows XP, I was testing all permutations of starting and running Visual Studio .NET and found that VS7JIT.EXE stopped running. After playing with it a bit, I found that two instances of VS7JIT.EXE actually run when starting Visual Studio .NET as the JIT debugger. One instance spawns the Visual Studio .NET IDE, and the other instance runs under the RPCSS DCOM server. On rare occasions, only during testing with the supplied implementation, I got the system into states where attempting to spawn VS7JIT.EXE would fail because the DCOM instance wouldn't start. I mainly ran into this problem when I worked on the code to reset the Debugger value in the AeDebug key. Once I settled on the current way of implementing DBGCHOOSER, I've run into the problem just once or twice and only when I set up test cases in which multiple process crashes were happening all at once. I wasn't able to track down the exact cause, but I've never seen this problem in normal work.
Post a comment