Using ADB to Debug the UNIX* Kernel Revised January, 1983 Samuel J. Leffler William N. Joy Computer Systems Research Group Department of Electrical Engineering and Computer Science University of California, Berkeley Berkeley, California 94720 (415) 642-7780 _A_B_S_T_R_A_C_T This document describes the use of extensions made to the 4.1bsd release of the VAX* UNIX debugger _a_d_b for the purpose of debugging the UNIX kernel. It discusses the changes made to allow standard _a_d_b commands to function properly with the kernel and introduces the basics necessary for users to write _a_d_b command scripts which may be used to augment the standard _a_d_b command set. The examination techniques described here may be applied to running systems, as well as the post- mortem dumps automatically created by the _s_a_v_e_c_o_r_e(8) program after a system crash. The reader is expected to have at least a passing fam- iliarity with the debugger command language. _________________________ *UNIX is a Trademark of Bell Laboratories. *DEC and VAX are trademarks of Digital Equipment Cor- poration. May 13, 1987 Using ADB on the UNIX Kernel Introduction _1. _I_N_T_R_O_D_U_C_T_I_O_N Modifications have been made to the standard VAX UNIX debugger _a_d_b to simplify examination of post-mortem dumps automatically generated following a system crash. These changes may also be used when examining UNIX in its normal operation. This document serves as an introduction to the _u_s_e of these facilities, and should not be construed as a description of _h_o_w _t_o _d_e_b_u_g _t_h_e _k_e_r_n_e_l. _1._1. _I_n_v_o_c_a_t_i_o_n When examining the UNIX kernel a new option, -k, should be used, e.g. adb -k /vmunix /dev/mem This flag causes _a_d_b to partially simulate the VAX virtual memory hardware when accessing the _c_o_r_e file. In addition the internal state maintained by the debugger is initialized from data structures maintained by the UNIX kernel expli- citly for debugging|=. A post-mortem dump may be examined in a similar fashion, adb -k vmunix.? vmcore.? where the appropriate version of the saved operating system image and core dump are supplied in place of ``?''. _1._2. _E_s_t_a_b_l_i_s_h_i_n_g _C_o_n_t_e_x_t During initialization _a_d_b attempts to establish the context of the ``currently active process'' by examining the value of the kernel variable _m_a_s_t_e_r_p_a_d_d_r. This variable contains the virtual address of the process context block of the last process which was set executing by the _S_w_t_c_h rou- tine. _M_a_s_t_e_r_p_a_d_d_r normally provides sufficient information to locate the current stack frame (via the stack pointers found in the context block). By locating the VAX process context block for the process, _a_d_b may then perform virtual to physical address translation using that process's in-core page tables. When examining post-mortem dumps locating the most recent stack frame of the ``currently active process'' is nontrivial. This is due to the different ways in which the _________________________ |= If the -k flag is not used when invoking _a_d_b the user must explicitly calculate virtual addresses. With the -_k option _a_d_b interprets page tables to automatically perform virtual to physical address translation. May 13, 1987 Using ADB on the UNIX Kernel- 2 - Introduction VAX may save state after a nonrecoverable error. Crashes may or may not be ``clean'' (i.e. the top of the interrupt stack contains the process's kernel mode stack pointer and program counter); an ``unclean'' crash will occur, for instance, if the interrupt stack overflows. Thus, one must manually try one of two possible techniques to get a stack trace from a post-mortem dump. If the crash was clean the current stack pointer is present in the restart parameter block, at _s_c_b-_4 (or _r_p_b+1fc), and the command *(scb-4)$c will generate a stack trace all the way from the kernel to the top of the user process's stack (e.g. to the _m_a_i_n rou- tine in the user process which was running). Otherwise, one must scan through the interrupt stack looking for the stack frame. This is usually indicated by a zero longword entry (the procedure call handler) followed by a longword entry with bit 29 set (indicating the call frame was generated as a result of a ``calls'' instruction). intstack/X Once the stack pointer has been located, the command .$c will generate a stack trace. An alternate method may be used when a trace of a particular process is required, see section 2.3. May 13, 1987 Using ADB on the UNIX Kernel- 3 - Command Scripts _2. _A_D_B _C_O_M_M_A_N_D _S_C_R_I_P_T_S _2._1. _E_x_t_e_n_d_i_n_g _t_h_e _F_o_r_m_a_t_t_i_n_g _F_a_c_i_l_i_t_i_e_s Once the process context has been established, the com- plete _a_d_b command set is available for interpreting data structures. In addition, a number of _a_d_b scripts have been created to simplify the structured printing of commonly referenced kernel data structures. The scripts normally reside in the directory /_u_s_r/_l_i_b/_a_d_b, and are invoked with the ``$<'' operator. (A later table lists the ``standard'' scripts.) As an example, consider the following listing which contains a dump of a faulty process's state (our typing is shown emboldened). % adb -k vmunix.17 vmcore.17 sbr 8001d064 slr d9c p0br 800efa00 p0lr 34 p1br 7f8efe00 p1lr 1ffff2 *(intstack-4)$c _boot() from 80004025 _boot(0,4) from 80004025 _panic(80021185) from 800057e2 _soreceive(8017478c,0) from 80007c90 _read() from 800098d7 _syscall() from 8000b6e2 _Xsyscall(3,7fffe834,258) from 80000f64 ?() from c1c ?() from 26a ?(0,7fffef18,7fffef1c) from 1d3 ?() from 2f 800021185/s _icpreg+99: receive u$l ,#l Place the value of the ``link'' in the _a_d_b variable ``4 *nproc>l *proc>f $l f ,#