Bramz' warehouse

From my previous post, you might know about my recent interest in minidumps to debug remote applications. Since then, I’ve been trying to implement an automatic and unattended crash dump facilitiy in Lass. On the Win32 platform, the key functions are SetUnhandledExceptionFilter in kernel32.lib and MiniDumpWriteDump in dbghelp.lib. The former lets you set a function to handle any exceptions that boil up from the application unhandled. The latter is used in that function to actually produce the minidump.

An interesting article on the subject is debugging custom filters for unhandled exceptions, especially the part about patching the SetUnhandledExceptionFilter function once you’ve set your filter, to prevent other modules (and even the runtime library) to set their own exception filters and thereby disabling yours. They do that by dynamically replacing the first instructions of that function by:

xor eax, eax
ret 0x4

Crude, but effective.

Once I’ve got it working on Win32, I started looking around for ways to deal with crashes in a similar way on the Linux platform. This has lead me to the discovery of Google-Breakpad (formely known as Airbag). It is a small open-source library that does exactly what I need: cross platform generation of minidumps on windows, mac and linux. It includes client and server parts, the client existing of a handler and sender. The handler is the actual exception handler that generates the minidump, the sender can be attached to the handler as a callback, in order to submit the minidump to a (public) server. Breakpad does not include an complete functional server, but it includes an minidump processor that converts the minidumps in human-readable stack walk, using local (local to the server) available symbol information. The benefit of doing this conversion on a server, in contrast to GNOME’s Bug-Buddy and Apple’s CrashReporter, is that the symbol files do not have to be shipped to the customer.

Breakpad surely looks like a promising tool that could be used in many (open-source) projects. Currently, there’s a mutual effort to include Breakpad in the next Firefox, replacing the closed source Talkback. This will certainly be a good test case, tremendously improving the library with experiences from a large scale project. For me, I certainly will try to include Breakpad in our software as well. There’s one part missing though: how to set up a public server that can receive the minidumps, automatically process them and inform me of the result. A few hints about deploying such a server may be found on the Mozilla wiki.

This entry was posted on Sunday, June 17th, 2007 at 8:15 am and is filed under coding. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.