Debugging kernel problems

Greg Lehey

Intended Audience:

Kernel developers and experienced system adminstrators. The tutorial assumes a good working knowledge of UNIX systems administration and the C programming language. A knowledge of kernel internals will be helpful, but is not essential. Depending on prior experience, participants will learn to locate and either fix or report kernel bugs.

Description:

This tutorial will show debugging techniques on live systems. The operating system for most of the tutorial will be FreeBSD, but it will explain the (relatively small) differences in NetBSD and OpenBSD. The course material, over 100 pages long, is a draft of a forthcoming book on the subject.

Topics:

o How and why kernels fail.
o Understanding log files: dmesg, /var/log/messages.
o Using common tools for debugging a running system: ps, netstat, top.
o Building a kernel with debugging support: the options.
o Preparing for dumps: dumpon, savecore.
o Demonstration: panicing and dumping a system.
o Preliminary dump analysis (non-technical, without sources).
o The assembler-level view of a C program.
o Introduction to the kernel source tree.
o Analysing panic dumps with gdb (technical).
o On-line kernel debuggers: ddb, gdb.
o Remote debugging with gdb, using serial lines, Ethernet and FireWire.
o Debugging a running system with gdb.
o If time permits, a demonstration of some of these topics on NetBSD and OpenBSD systems.

Tutorial (Friday)

All

Double

Greg Lehey is a FreeBSD and NetBSD developer and an ex-member of the FreeBSD core team. He has been in the computer industry for 30 years, most of them spent in Germany, in which time he has performed most jobs, ranging from kernel development to product management, from systems programming to systems administration, from processing satellite data to programming petrol pumps, from the production of CD-ROMs of ported free software to DSP instruction set design. He is the author of "Porting UNIX Software" (O'Reilly and Associates, 1995) and "The Complete FreeBSD" (O'Reilly and Associates, 2003).