linux poison RSS
linux poison Email

Auto reboot after kernel panic

When a kernel encounters certain errors it calls the "panic" function which results from a unrecoverable error. This panic results in LKCD (Linux Kernel Crash Dump) initiating a kernel dump where kernel memory is copied out to the pre-designated dump area. The dump device is configured as primary swap by default. The kernel is not completely functional at this point, but there is enough functionality to copy memory to disk. When the system boots back up, it checks for a new crash dump. If a new crash dump is found it is copied from the dump location to the file system, "/var/log/dump" directory by default. After copying the image, the system continues to boot normally and forensics can be performed at a later date.

By default after a kernel panic, system just waits there for a restart.  This is because of the value set on "kernel.panic" parameter.

# cat /proc/sys/kernel/panic

To disable this and make the Linux OS reboot after a kernel panic, we have to set an integer value greater than zero to the paramter "kernel.panic", where the value is the number of seconds to wait before a automatic reboot.  For example , if you set it to "10" , then the system waits for 10 seconds before automatic reboot. To make this permanent, edit /etc/sysctl.conf and add following like at end of the file.

kernel.panic = 10


Matt Simmons said...

I have very mixed feeling about this. I suspect that, before I was comfortable with this, I would write a script to check for the dump file, and if it exists, send me an email letting me know that it crashed because of a kernel panic.

It's one thing to debug a machine that just reboots on its own. It's an entirely different one to debug a machine that kernel panics. Making them look similar makes me feel all funny.

Anonymous said...

/var/log/dump doesn't exist in any linux distro I know.

DevOps said...

this directory "/var/log/dump" will automatically get created on kernel dump

Post a Comment

Related Posts with Thumbnails