This page is going to be about LinuxKernel development.
There are useful tools for kernel development available such as Ostra and Sparse.
Today most KernelDevelopment is done with Git. See KernelDevelopmentWithGit for more details.
To debug a kernel crash you can use objdump to show the assembler for the routine shown and if debug symbols from kernel hacking menu are turned on then you can see the C code also. There will be a hex offset in the crash output and just use that to find the valid line of code/assembler. For example: objdump -r -S -l --disassemble net/dccp/ipv4.o
NB. You need to be at the top level of the kernel tree for this to pick up your C files.
If you don't have access to the code you can also debug on some crash dumps e.g. crash dump output as shown by Dave Miller.
EIP is at ip_queue_xmit+0x14/0x4c0
...
Code: 44 24 04 e8 6f 05 00 00 e9 e8 fe ff ff 8d 76 00 8d bc 27 00 00
00 00 55 57 56 53 81 ec bc 00 00 00 8b ac 24 d0 00 00 00 8b 5d 08
<8b> 83 3c 01 00 00 89 44 24 14 8b 45 28 85 c0 89 44 24 18 0f 85
Put the bytes into a "foo.s" file like this:
.text
.globl foo
foo:
.byte .... /* bytes from Code: part of OOPS dump */
Compile it with "gcc -c -o foo.o foo.s" then look at the output
of "objdump --disassemble foo.o".
Output:
ip_queue_xmit:
push %ebp
push %edi
push %esi
push %ebx
sub $0xbc, %esp
mov 0xd0(%esp), %ebp ! %ebp = arg0 (skb)
mov 0x8(%ebp), %ebx ! %ebx = skb->sk
mov 0x13c(%ebx), %eax ! %eax = inet_sk(sk)->opt
A couple of useful articles for kernel development from IBM:
http://www-128.ibm.com/developerworks/linux/edu/l-dw-linux-kernelhack1-i.html
http://www-128.ibm.com/developerworks/linux/edu/l-dw-linux-kernelhack2-i.html
A useful cross reference of the LinuxKernel source code can be found at http://lxr.linux.no/
In theory you can use DistCC to do KernelDevelopment although it can be problematic.
Here is a script that I use to copy my new kernel to other remote machines:
#! /bin/bash SRC=$HOME/linuxsrc/dccpwork VER=2.6.14-rc1 rm $HOME/tmp/lib/modules/$VER/build rm $HOME/tmp/lib/modules/$VER/source scp $SRC/System.map root@$1:/boot/System.map-$VER scp $SRC/arch/i386/boot/bzImage root@$1:/boot/vmlinuz-$VER scp -r $HOME/tmp/lib root@$1:/
