Linux 核心除錯 kdump vmcore

linux 核心除錯 kdump vmcore

linux核心傳送崩潰時，kdump會生成乙個核心轉儲檔案vmcore。可以通過分析vmcore分析出核心崩潰的原因。

crash是乙個被廣泛應用的核心奔潰轉儲檔案分析工具。

使用crash除錯核心轉儲檔案，需要安裝crash工具和核心除錯工具kernel-debuginfo。

安裝kdump

crash kexec-tools 一般在系統映象檔案中就有相對應的rpm包

配置kdump：

vim /boot/grub/menu.lst：設定 crashkernel=auto

vim /etc/kdump.conf： path /var/crash （core檔案產生的目錄）

啟動kdump：

service kdump start

分析vmcore

crash /usr/lib/debug/lib/modules/3.10.0-957.el7.x86_64/vmlinux vmcore

crash> bt pid: 7473 task: ffff9027d874bf40 cpu: 0 command: "cat" #0 [ffff9026d0ea3638] machine_kexec at ffffffffbd060b2a #1 [ffff9026d0ea3698] __crash_kexec at ffffffffbd113402 #2 [ffff9026d0ea3768] crash_kexec at ffffffffbd1134f0 #3 [ffff9026d0ea3780] oops_end at ffffffffbd717778 #4 [ffff9026d0ea37a8] no_context at ffffffffbd706f98 #5 [ffff9026d0ea37f8] __bad_area_nosemaphore at ffffffffbd70702f #6 [ffff9026d0ea3848] bad_area_nosemaphore at ffffffffbd7071a0 #7 [ffff9026d0ea3858] __do_page_fault at ffffffffbd71a730 #8 [ffff9026d0ea38c0] do_page_fault at ffffffffbd71a925 #9 [ffff9026d0ea38f0] page_fault at ffffffffbd716768 [exception rip: strcmp+32] rip: ffffffffbd353d20 rsp: ffff9026d0ea39a0 rflags: 00010202 rax: 000000000000002f rbx: ffff90240da5a080 rcx: 0000000000000000 rdx: 0000000000000000 rsi: 0000000000000001 rdi: ffff9026cd27fc11 rbp: ffff9026d0ea39a0 r8: 00000000004b1de2 r9: ffff9026cd27fc10 r10: ffff90253fc01d00 r11: ffffc0428c349fc0 r12: 0000000000000001 r13: ffff9026cd27fc10 r14: 0000000000000001 r15: ffff9027c16f1580 orig_rax: ffffffffffffffff cs: 0010 ss: 0018 #10 [ffff9026d0ea39a8] send_log at ffffffffc0c22fd5 [xx] #11 [ffff9026d0ea3ac0] user_file at ffffffffc0c0c571 [xx] #12 [ffff9026d0ea3f00] sys_open at ffffffffc0c56670 [xxt] #13 [ffff9026d0ea3f50] system_call_fastpath at ffffffffbd71f7d5 rip: 00007f2e860c2a30 rsp: 00007fff6d8755a8 rflags: 00010202 rax: 0000000000000002 rbx: 00007fff6d875868 rcx: 000000000060bc60 rdx: 1fffffffffff0000 rsi: 0000000000000000 rdi: 00007fff6d876293 rbp: 0000000000001000 r8: 0000000000000000 r9: 0000000000000000 r10: 00007fff6d875020 r11: 0000000000000246 r12: 0000000000402644 r13: 0000000000010000 r14: 0000000000000000 r15: 0000000000000000 orig_rax: 0000000000000002 cs: 0033 ss: 002b crash> dis -l ffffffffbd353d20 /usr/src/debug/kernel-3.10.0-862.el7/linux-3.10.0-862.el7.x86_64/lib/string.c: 253 0xffffffffbd353d20 : cmp -0x1(%rsi),%al crash> dis -l ffffffffc0c22fd5

0xffffffffc0c22fd5 : test %eax,%eax

可以看到問題出現在send_to_user_identify_log+901

901 = 0x385

反彙編xx.ko

objdump -s xx.ko > tmp

開啟tmp檔案，檢視彙編**，找到send_log+0x385位置

16fc5:       4d 8b 6f 50             mov    0x50(%r15),%r13
else if (result > 0) node = node->rb_right;
16fc9:       48 8b 73 50             mov    0x50(%rbx),%rsi
if (result < 0) node = node->rb_left;
16fcd:       4c 89 ef                mov    %r13,%rdi
16fd0:       e8 00 00 00 00          callq  16fd5 16fd5:       85 c0                   test   %eax,%eax
16fd7:       78 57                   js     17030 else if (result > 0) node = node->rb_right;
16fd9:       75 43                   jne    1701e if (result < 0) node = node->rb_left;
16fdb:       4d 85 ed                test   %r13,%r13
16fde:       66 90                   xchg   %ax,%ax
16fe0:       74 10                   je     16ff2 16fe2:       4c 89 ef                mov    %r13,%rdi
else if (result > 0) node = node->rb_right;
16fe5:       e8 00 00 00 00          callq  16fea result = req->op_result - data->req.op_result;
16fea:       49 c7 47 50 00 00 00    movq   $0x0,0x50(%r15)
16ff1:       00
if (result < 0) node = node->rb_left;

可以看出問題出現在

if (result < 0) node = node->rb_left;

crash 簡單的命令

crash> help

files mach repeat timer

alias foreach mod runq tree

ascii fuser mount search union

bt gdb net set vm

btop help p sig vtop

dev ipcs ps struct waitq

dis irq pte swap whatis

eval kmem ptob sym wr

exit list ptov sys q

extend log rd task

Linux 核心除錯 kdump vmcore

linux 核心除錯

linux 核心除錯（一）

linux核心崩潰除錯

Linux 核心除錯 kdump vmcore

linux 核心除錯

linux 核心除錯（一）

linux核心崩潰除錯

相關推薦