How to read, understand, analyze, and debug a Linux kernel panic?

c linux debugging linux-kernel panic
0x90 picture 0x90 · Nov 20, 2012 · Viewed 68.1k times · Source

Consider the following linux kernel dump stack trace, you can trigger a panic from the kernel source code by calling panic("debugging a linux kernel panic");:

[<001360ac>] (unwind_backtrace+0x0/0xf8) from [<00147b7c>] (warn_slowpath_common+0x50/0x60)
[<00147b7c>] (warn_slowpath_common+0x50/0x60) from [<00147c40>] (warn_slowpath_null+0x1c/0x24)
[<00147c40>] (warn_slowpath_null+0x1c/0x24) from [<0014de44>] (local_bh_enable_ip+0xa0/0xac)
[<0014de44>] (local_bh_enable_ip+0xa0/0xac) from [<0019594c>] (bdi_register+0xec/0x150)
  • In unwind_backtrace+0x0/0xf8 what the +0x0/0xf8 stands for?
  • How can I see the C code of unwind_backtrace+0x0/0xf8?
  • How to interpret the panic's content?

Answer

iabdalkader picture iabdalkader · Nov 20, 2012

It's just an ordinary backtrace, those functions are called in reverse order (first one called was called by the previous one and so on):

unwind_backtrace+0x0/0xf8
warn_slowpath_common+0x50/0x60
warn_slowpath_null+0x1c/0x24
ocal_bh_enable_ip+0xa0/0xac
bdi_register+0xec/0x150

The bdi_register+0xec/0x150 is the symbol + the offset/length there's more information about that in Understanding a Kernel Oops and how you can debug a kernel oops. Also there's this excellent tutorial on Debugging the Kernel

Note: as suggested below by Eugene, you may want to try addr2line first, it still needs an image with debugging symbols though, for example

addr2line -e vmlinux_with_debug_info 0019594c(+offset)

Related questions

How do I pass a command line argument while starting up GDB in Linux?

I have to debug a program that has errors in it as part of my assignment. However, I must first pass command line arguments in order to solve this problem. I do: gdb -tui InsertionSortWithErrors which works, but after that …

Read More
How to solve "ptrace operation not permitted" when trying to attach GDB to a process?

I'm trying to attach a program with gdb but it returns: Attaching to process 29139 Could not attach to process. If your uid matches the uid of the target process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or …

Read More
How to make backtrace()/backtrace_symbols() print the function names?

The Linux specific backtrace() and backtrace_symbols() allows you to produce a call trace of the program. However, it only prints function addresses, not their names for my program. How can I make them print the function names as well ? …

Read More