1.\" $OpenBSD: ddb.4,v 1.58 2009/02/22 15:10:41 jsing Exp $ 2.\" $NetBSD: ddb.4,v 1.5 1994/11/30 16:22:09 jtc Exp $ 3.\" 4.\" Mach Operating System 5.\" Copyright (c) 1991,1990 Carnegie Mellon University 6.\" All Rights Reserved. 7.\" 8.\" Permission to use, copy, modify and distribute this software and its 9.\" documentation is hereby granted, provided that both the copyright 10.\" notice and this permission notice appear in all copies of the 11.\" software, derivative works or modified versions, and any portions 12.\" thereof, and that both notices appear in supporting documentation. 13.\" 14.\" CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" 15.\" CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND FOR 16.\" ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. 17.\" 18.\" Carnegie Mellon requests users of this software to return to 19.\" 20.\" Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU 21.\" School of Computer Science 22.\" Carnegie Mellon University 23.\" Pittsburgh PA 15213-3890 24.\" 25.\" any improvements or extensions that they make and grant Carnegie Mellon 26.\" the rights to redistribute these changes. 27.\" 28.Dd $Mdocdate: February 22 2009 $ 29.Dt DDB 4 30.Os 31.Sh NAME 32.Nm ddb 33.Nd kernel debugger 34.Sh DESCRIPTION 35The 36.Nm 37debugger provides a means for debugging the kernel, 38and analysing the kernel after a system crash ("panic"), with a 39.Xr gdb 1 Ns \&-like 40syntax. 41.Pp 42.Nm 43is invoked upon a kernel panic when the 44.Xr sysctl 8 45.Va ddb.panic 46is set to 1. 47It may be invoked from the console when the sysctl 48.Va ddb.console 49is set to 1, using any of the following methods: 50.Pp 51.Bl -dash -offset 3n 52.It 53Using the key sequence 54.Li Ctrl-Alt-Esc . 55.It 56Sending a 57.Dv BREAK 58when using a serial console. 59.It 60Writing to the sysctl 61.Va ddb.trigger . 62.It 63For i386 and amd64 architectures, 64using the key sequence 65.Li Ctrl-Alt-Delete 66when the sysctl 67.Va machdep.kbdreset 68is set to 2. 69.El 70.Pp 71.Nm 72prompts for commands on the console with: 73.Pp 74.Dl ddb\*(Gt 75.Pp 76The general syntax of a 77.Nm 78command is: 79.Bd -ragged -offset indent 80.Ar command 81.Oo Ic / Ns Ar modifiers Oc " " 82.Oo Ar address Oc Ns 83.Op Ic \&, Ns Ar count 84.Ed 85.Pp 86To save typing, 87.Nm 88makes use of a context inferred from previous commands. 89In this context, 90the current location is called 91.Va dot . 92.\" The 93.\" .Va dot 94.\" is displayed with 95.\" a hexadecimal format at a prompt. 96The 97.Ic examine , 98.Ic search , 99and 100.Ic write 101commands update 102.Va dot 103to be that of the last address 104examined or the last location modified, and 105have intuitive effects on 106.Va next 107and 108.Va prev . 109All the other commands do not change 110.Va dot , 111and set 112.Va next 113to be the same. 114(See 115.Sx VARIABLES . ) 116.Pp 117.\" Specifying 118.\" .Ar address 119.\" in a command sets 120.\" .Va dot . 121An expression can be used in place of 122.Ar address 123(see 124.Sx EXPRESSIONS ) . 125Omitting 126.Ar address 127in a command uses the last value of 128.Va dot . 129A missing 130.Ar count 131is taken to be 1 for printing commands or \*(If 132for stack traces. 133Entering a blank line causes the last command to be repeated using 134.Va next 135in place of 136.Ar address , 137a 138.Ar count 139of 1, and no modifiers. 140.Pp 141.Nm 142has a feature like 143.Xr more 1 144for the output. 145If the number of lines output in response to one command exceeds the number 146set in the 147.Va $lines 148variable, it displays the message 149.Ql "--db_more--" 150and waits for a response. 151.Pp 152The valid responses are: 153.Pp 154.Bl -tag -width 10n -offset indent -compact 155.It Aq Ic space 156One more page. 157.It Aq Ic return 158One more line. 159.It Ic q 160Abort the current command, and return to the command input mode. 161.El 162.Pp 163The following command line editing keys are provided: 164.Pp 165.Bl -tag -width 10n -offset indent -compact 166.It Ic ^b 167back one character 168.It Ic ^f 169forward one character 170.It Ic ^a 171beginning of line 172.It Ic ^e 173end of line 174.It Ic ^w 175erase word back 176.It Ic ^h | Aq Ic del 177erase previous character 178.It Ic ^d 179erase next character 180.It Ic ^k 181delete to end of line 182.It Ic ^u 183delete line 184.It Ic ^p 185previous in command history 186.It Ic ^n 187next in command history 188.It Ic ^r 189redraw line 190.It Ic ^t 191exchange the two characters to the left of the cursor 192.El 193.Sh COMMANDS 194The following commands may be typed at the 195.Ql ddb\*(Gt 196prompt. 197Some commands consist of more than one word, and if only the first word 198or words are entered, the possible alternatives to complete the command 199are displayed and no other action is performed. 200.Bl -tag -width 10n 201.\" -------------------- 202.It Ic help 203List the available commands. 204.\" -------------------- 205.It Xo 206.Oo Ic e Oc Ns 207.Ic x Ns Op Ic amine 208.Op Cm /bhlaAxzodurcsmiI 209.Op Ar addr Ns 210.Op Ic \&, Ns Ar count 211.Xc 212Display the contents at address 213.Ar addr 214according to the formats in the modifier. 215Multiple modifier formats display multiple locations. 216If no format is specified, the last formats specified for this command 217are used. 218.Pp 219The format characters are: 220.Pp 221.Bl -tag -width 4n -compact 222.It Cm /b 223look at by bytes (8 bits) 224.It Cm /h 225look at by half words (16 bits) 226.It Cm /l 227look at by long words (32 bits) (default) 228.It Cm /q 229look at by long longs (64 bits) (only available on 64-bit platforms) 230.It Cm /a 231print the location being displayed 232.It Cm /A 233print the location with a line number if possible 234.It Cm /x 235display in unsigned hex 236.It Cm /z 237display in signed hex 238.It Cm /o 239display in unsigned octal 240.It Cm /d 241display in signed decimal 242.It Cm /u 243display in unsigned decimal 244.It Cm /r 245display in current radix, signed 246.It Cm /c 247display low 8 bits as a character. 248Non-printing characters are displayed as an octal escape code (e.g., '\\000'). 249.It Cm /s 250display the null-terminated string at the location. 251Non-printing characters are displayed as octal escapes. 252.It Cm /m 253display in unsigned hex with character dump at the end of each line. 254The location is also displayed in hex at the beginning of each line. 255.It Cm /i 256display as an instruction 257.It Cm /I 258display as an alternate format instruction depending on the 259machine: 260.Pp 261.Bl -tag -width powerpc_ -compact 262.It alpha 263Print affected register contents for every instruction. 264.It amd64 , 265.It i386 266Do not skip padding to the next long word boundary for unconditional jumps. 267.It m68k 268Use 269.Tn Motorola 270assembly syntax. 271.It m88k 272Decode instructions for the opposite CPU model (e.g. m88110 when running on an 273m88100 processor). 274.It vax 275Don't assume that each external label is a procedure entry mask. 276.El 277.El 278.Pp 279The value of 280.Va next 281is set to the 282.Ar addr 283plus the size of the data examined. 284.\" -------------------- 285.It Xo 286.Ic print 287.Op Cm /axzodurc 288.Op Ar addr Op Ar addr ... 289.Xc 290Print each 291.Ar addr 292according to the modifier character. 293The valid modifiers are a subset of those from the 294.Ic examine 295command, and act as described there. 296If no modifier is specified, the last one specified in a 297previous use of 298.Ic print 299is used. 300The 301.Ar addr 302argument 303can be a string, and it is printed as a literal. 304.Pp 305For example, 306.Bd -literal -offset indent 307print/x "eax = " $eax "\enecx = " $ecx "\en" 308.Ed 309.Pp 310will print something like this: 311.Bd -literal -offset indent 312eax = xxxxxx 313ecx = yyyyyy 314.Ed 315.\" -------------------- 316.\" .It Xo Ic w Ns Op Cm /bhl 317.\" .Op Ar addr 318.\" .Ar expr Op expr ... 319.\" .Xc 320.It Xo 321.Ic w Ns Op Ic rite 322.Op Cm /bhl 323.Op Ar addr 324.Ar expr Op Ar expr ... 325.Xc 326Write the value of each 327.Ar expr 328expression at succeeding locations start at 329.Ar addr . 330The write unit size can be specified using one of the modifiers: 331.Pp 332.Bl -tag -width 4n -offset indent -compact 333.It Cm /b 334byte (8 bits) 335.It Cm /h 336half word (16 bits) 337.It Cm /l 338long word (32 bits) (default) 339.El 340.Pp 341The value of 342.Va next 343is set to 344.Ar addr 345plus the size of values written. 346.Pp 347.Sy Warning: 348since there is no delimiter between expressions, the 349command may not parse as you expect. 350It is best to enclose each expression in parentheses. 351.\" -------------------- 352.It Xo Ic set 353.Ic $ Ns Ar name 354.Op Ic = 355.Ar expr 356.Xc 357Set the named variable or register with the value of 358.Ar expr . 359Valid variable names are described below. 360.It Ic boot Ar how 361Reboot the machine depending on 362.Ar how : 363.Pp 364.Bl -tag -width "boot poweroff" -compact 365.It Ic boot sync 366Sync disks and reboot. 367.It Ic boot crash 368Dump core and reboot. 369.It Ic boot dump 370Sync disks, dump core and reboot. 371.It Ic boot halt 372Just halt. 373.It Ic boot reboot 374Just reboot. 375.It Ic boot poweroff 376Power down the machine whenever possible; if it fails, just halt. 377.El 378.\" -------------------- 379.It Xo 380.Ic break 381.Op Ar addr Ns 382.Op Ic \&, Ns Ar count 383.Xc 384Set a break point at 385.Ar addr . 386If 387.Ar count 388is supplied, 389.Nm 390allows the breakpoint to be silently hit 391.Ar ( count No \- 1 ) 392times before stopping at the 393break point. 394.Pp 395If the break point is successfully set, a break point number is 396displayed, in the form 397.Ic # Ns Ar number . 398This can later be used in deleting the break point 399or for adding conditions to it. 400.\" -------------------- 401.\" .It Xo Ic d 402.\" .Op Ar addr | Ic # Ns Ar number 403.\" .Xc 404.It Xo 405.Ic d Ns Op Ic elete 406.Op Ar addr | Ic # Ns Ar number 407.Xc 408Delete the break point set with the 409.Ic break 410command. 411.\" -------------------- 412.\" .It Xo Ic s Ns Op Cm /p 413.\" .Op Ic \&, Ns Ar count 414.\" .Xc 415.It Xo 416.Ic s Ns Op Ic tep 417.Op Cm /p 418.Op Ic \&, Ns Ar count 419.Xc 420Single step 421.Ar count 422times. 423If the 424.Cm /p 425modifier is specified, print each instruction at each step. 426Otherwise, only print the last instruction. 427.Pp 428.Sy Warning: 429depending on machine type, it may not be possible to 430single-step through some low-level code paths or user space code. 431On machines with software-emulated single-stepping (e.g., pmax), 432stepping through code executed by interrupt handlers will probably 433do the wrong thing. 434.\" -------------------- 435.It Ic call Ar name Ns Xo 436.Ic \&( Ns Ar expr 437.Op Ic \&, Ar expr ... 438.Ic \&) 439.Xc 440Call the function named by 441.Ar name 442with the argument(s) listed in parentheses. 443Parentheses may be omitted if the function takes no arguments. 444The number of arguments is currently limited to 10. 445.\" -------------------- 446.\" .It Ic c Ns Op Cm /c 447.It Xo 448.Ic c Ns Op Ic ontinue 449.Op Cm /c 450.Xc 451Continue execution until a breakpoint or watchpoint. 452If the 453.Cm /c 454modifier is given, instructions are counted while executing. 455Some machines (e.g., pmax) also count loads and stores. 456.Pp 457.Sy Warning: 458when counting with 459.Cm /c , 460.Nm 461is really silently single-stepping. 462This means that single-stepping on low-level code may cause strange 463behavior. 464.\" -------------------- 465.It Xo 466.Ic watch 467.Ar addr 468.Op Ic \&, Ns Ar size 469.Xc 470Set a watchpoint for the region starting at 471.Ar addr . 472Execution stops and control returns to 473.Nm 474when an attempt is made to modify a watched region. 475The 476.Ar size 477argument defaults to 4. 478.Pp 479If you specify a wrong space address, the request is rejected 480with an error message. 481.Pp 482.Sy Warning: 483attempts to watch wired kernel memory 484may cause an unrecoverable error on some systems (e.g., i386). 485Watchpoints on user addresses work best. 486.\" -------------------- 487.It Ic dwatch Ar addr 488Delete the watchpoint at address 489.Ar addr 490that was previously set with a 491.Ic watch 492command. 493.\" -------------------- 494.It Xo 495.Ic hangman 496.Op Cm /s Ns Op Ic 0-9 497.Xc 498This is a tiny and handy tool for random kernel hangs analysis, of which its 499depth is controlled by the optional argument of the default value of five. 500It uses some sophisticated heuristics to spot the global symbol that 501caused the hang. 502Since the discovering algorithm is a probabilistic one, 503you may spend substantial time to figure the exact symbol name. 504This smart thing requires a little of your attention, the input it accepts 505is mostly of the same format as that of the famous 506.Xr hangman 6 507game, to which it, apparently, is obliged by the name. 508Hint: the 509.Xr nm 1 510utility might help. 511.\" -------------------- 512.It Ic until Op Cm /p 513Stop at the next 514.Qq call 515or 516.Qq return 517instruction. 518If the 519.Cm /p 520modifier is specified, 521.Nm 522prints the call nesting depth and the 523cumulative instruction count at each call or return. 524Otherwise, it stays silent until the matching return is hit. 525.\" -------------------- 526.It Ic match Op Cm /p 527Stop at the next matching return instruction. 528If the 529.Cm /p 530modifier is specified, 531.Nm 532prints the call nesting depth and the 533cumulative instruction count at each call or return. 534Otherwise, it remains mostly quiet. 535.\" -------------------- 536.It Ic next Op Cm /p 537The 538.Ic next 539command is a synonym for 540.Ic match . 541.\" -------------------- 542.It Xo 543.Ic trace 544.Op Cm /pu 545.Op Ar frameaddr Ns 546.Op Ic \&, Ns Ar count 547.Xc 548Show the stack trace. 549The 550.Cm /p 551modifier interprets the 552.Ar frameaddr 553argument as the PID of a process and shows the stack trace of 554that process. 555The 556.Cm /p 557modifier is not supported on all platforms. 558The 559.Cm /u 560modifier shows the stack trace of user space; 561if omitted, the kernel stack is traced instead. 562The 563.Ar count 564argument is the limit on the number of frames to be followed. 565If 566.Ar count 567is omitted, all frames are printed. 568.Pp 569.Sy Warning: 570user space stack trace is valid 571only if the machine dependent code supports it. 572.\" -------------------- 573.It Xo 574.Ic search 575.Op Cm /bhl 576.Op Ar addr 577.Ar value 578.Op Ar mask 579.Op Ic \&, Ns Ar count 580.Xc 581Search memory for a value beginning at 582.Ar addr . 583This command might fail in interesting 584ways if it doesn't find the searched-for value. 585This is because 586.Nm 587doesn't always recover from touching bad memory. 588The optional 589.Ar count 590argument limits the search. 591The modifiers are the same as those of the 592.Ic write 593command. 594.Pp 595The 596.Va next 597address is set to the address where 598.Ar value 599is found, or just after where the search area finishes. 600.\" -------------------- 601.It Ic show Ar what 602Displays various things, depending on 603.Ar what : 604.Bl -tag -width 4n 605.\" -------------------- 606.It Ic show breaks 607Prints a list of all breakpoints that have been set with the 608.Ic break 609command. 610.\" -------------------- 611.It Xo 612.Ic show buf 613.Op Cm /f 614.Ar addr 615.Xc 616Prints the 617.Li struct buf 618at 619.Ar addr . 620If the 621.Cm /f 622modifier is specified output will also include 623.Li softdep 624printout, if those are available. 625.\" -------------------- 626.It Ic show extents 627Prints a detailed list of all extents. 628.\" -------------------- 629.It Ic show malloc Op Ar addr 630Prints malloc debugging information if available. 631If an optional address is specified, only information about that address 632is printed. 633.\" -------------------- 634.It Xo 635.Ic show map 636.Op Cm /f 637.Ar addr 638.Xc 639Prints the 640.Li vm_map 641at 642.Ar addr . 643If the 644.Cm /f 645modifier is specified the complete map is printed. 646.\" -------------------- 647.It Xo 648.Ic show mount 649.Op Cm /f 650.Ar addr 651.Xc 652Prints the 653.Li struct mount 654at 655.Ar addr . 656If the 657.Cm /f 658modifier is specified prints out all 659.Li vnodes 660(see also 661.Cm show vnode ) 662and also all 663.Li bufs 664(see also 665.Cm show buf ) 666on all those vnodes. 667.\" -------------------- 668.It Xo 669.Ic show object 670.Op Cm /f 671.Ar addr 672.Xc 673Prints the 674.Li vm_object 675at 676.Ar addr . 677If the 678.Cm /f 679modifier is specified the complete object is printed. 680.\" -------------------- 681.It Xo 682.Ic show page 683.Op Cm /f 684.Ar addr 685.Xc 686Prints the 687.Li vm_page 688at 689.Ar addr . 690If the 691.Cm /f 692modifier is specified the complete page is printed. 693.\" -------------------- 694.It Ic show panic 695Prints the panic string. 696.\" -------------------- 697.It Xo 698.Ic show pool 699.Op Cm /clp 700.Ar addr 701.Xc 702Prints the 703.Li pool 704at 705.Ar addr . 706Valid modifiers: 707.Bl -tag -width xxx -compact 708.It Cm /c 709Print the cachelist and its statistics for this pool. 710.It Cm /l 711Print the log entries for this pool. 712.It Cm /p 713Print the pagelist for this pool. 714.El 715.\" -------------------- 716.It Ic show proc Op Ar addr 717Prints the 718.Li struct proc 719at 720.Ar addr . 721If an optional address is not specified 722.Li curproc 723is assumed. 724.\" -------------------- 725.It Ic show registers Op Cm /u 726Display the register set. 727If the 728.Cm /u 729modifier is specified, it displays user registers (or the currently 730saved registers) instead of the kernel's. 731Note: The 732.Cm /u 733modifier is not supported on every machine, in which case 734incorrect information may be displayed. 735.\" -------------------- 736.It Ic show uvmexp 737Displays a selection of uvm counters and statistics. 738.\" -------------------- 739.It Xo 740.Ic show vnode 741.Op Cm /f 742.Ar addr 743.Xc 744Prints the 745.Li struct vnode 746at 747.Ar addr . 748If the 749.Cm /f 750modifier is specified prints out all 751.Li bufs 752(see also 753.Cm show buf ) 754currently attached to this 755.Li vnode . 756.\" -------------------- 757.It Xo 758.Ic show nfsreq 759.Op Cm /f 760.Ar addr 761.Xc 762Prints the 763.Li struct nfsreq 764at 765.Ar addr. 766If the 767.Cm /f 768modifier is specified prints out additional 769information as well. 770.\" -------------------- 771.It Ic show watches 772Displays all watchpoints set with the 773.Ic watch 774command. 775.\" -------------------- 776.It Ic show all procs Op Cm /anw 777Display information on all processes. 778.Pp 779.Bl -tag -width foo -compact 780.It Cm /n 781(Default) Show process information in a 782.Xr ps 1 Ns \&-like 783format. 784Information printed includes process ID, parent 785process ID, process group, UID, process status, process flags, process 786command name, and process wait channel message. 787.It Cm /a 788Shows the kernel virtual addresses of each process' 789proc structure, u-area, and vmspace structure. 790The vmspace address is also the address of the process' 791.Li vm_map 792structure 793and can be used in the 794.Ic show map 795command. 796.It Cm /w 797Shows each process' PID, command, system call emulation, 798wait channel address, and wait channel message. 799.El 800.\" -------------------- 801.It Ic show all callout 802Display the contents of the callout table. 803.\" -------------------- 804.It Ic show all pools Op Cm /a 805Display information about all system pools in a format similar to 806.Xr vmstat 8 . 807.Pp 808.Bl -tag -width foo -compact 809.It Cm /a 810Displays 811.Dq interesting 812address information. 813.El 814.\" -------------------- 815.It Ic show all mounts Op Cm /f 816Display information on all mounted filesystems. 817.Pp 818.Bl -tag -width foo -compact 819.It Cm /f 820For each filesystem, list all its struct vnode addresses. 821These addresses can be used in the 822.Ic show vnode 823command. 824.El 825.\" -------------------- 826.It Ic show all nfsreq 827Display a list of pointers to all outstanding nfs requests. 828.Pp 829.El 830.It Ic callout 831A synonym for the 832.Ic show all callout 833command. 834.\" -------------------- 835.It Ic ps Op Cm /anw 836A synonym for 837.Ic show all procs . 838.\" -------------------- 839.El 840.Sh VARIABLES 841.Nm 842denotes registers and variables by 843.Ic $ Ns Va name . 844Register names can be found with the 845.Ic show registers 846command. 847.Pp 848Some variable names are suffixed with numbers, and some may have a modifier 849following a colon immediately after the variable name. 850For example, register variables can have the 851.Ql :u 852modifier to indicate a 853user register (e.g., 854.Ql $eax:u ) . 855.Pp 856Built-in debugger variables currently supported are: 857.Bl -tag -width 10n -compact -offset indent 858.It Va $radix 859Input and output radix. 860.It Va $maxoff 861Addresses are printed as 862.Ar symbol Ns Li + Ns Ar offset 863unless 864.Ar offset 865is greater than 866.Va $maxoff . 867.It Va $maxwidth 868The width of the displayed lines. 869.It Va $lines 870The number of lines to page. 871This is used by the 872.Dq more 873feature. 874.It Va $tabstops 875Tab stop width. 876.It Va $log 877Controls whether the output of ddb will also appear in the system message 878buffer. 879.El 880.Pp 881These variables can also be controlled outside ddb via the 882.Sq ddb 883.Xr sysctl 8 884hierarchy. 885.Sh EXPRESSIONS 886Almost all expression operators in C are supported except for 887.Ql ~ , 888.Ql ^ , 889and unary 890.Ql & . 891Special rules for expressions in 892.Nm 893are: 894.Bl -tag -width 15n -compact -offset indent 895.It Ar identifier 896The name of a symbol. 897It is translated to the address (or value) of the symbol. 898.Ql \&. 899and 900.Ql \&: 901can be used in the identifier. 902The following can be accepted as an identifier, 903if supported by an object format dependent routine: 904.Bl -item -offset indent -compact 905.It 906.Sm off 907.Oo Ar filename Li \&: Oc Ar func 908.Op \&: Ar linenumber 909.It 910.Op Ar filename \&: 911.Ar variable 912.It 913.Ar filename 914.Op \&: Ar linenumber 915.Sm on 916.El 917The symbol may be prefixed with 918.Sq Ar symboltablename Ns :: 919(e.g., 920.Ql emulator::mach_msg_trap ) 921to specify other than kernel symbols. 922.It Ar number 923The radix is determined by the first two letters: 924.Ql 0x : 925hex, 926.Ql 0o : 927octal, 928.Ql 0t : 929decimal, otherwise, the value of 930.Va $radix 931is used. 932.It Li \&. 933.Va dot : 934the current address. 935.It Li + 936.Va next : 937the next address. 938.It Li .. 939The address of the start of the last line examined. 940Unlike 941.Va dot 942or 943.Va next , 944this is only changed by the 945.Ic examine 946or 947.Ic write 948command. 949.It Li ' 950The last address explicitly specified. 951.It Li $ Ns Ar variable 952The value of a register or variable. 953The name may be followed by a 954.Ql \&: 955and modifiers as described above with 956.Ar identifier . 957.It Ar expr Li # Ar expr 958A binary operator which rounds up the left hand side to the next 959multiple of right hand side. 960.It Li * Ns Ar expr 961Indirection. 962It may be followed by a ':' and modifiers as described above. 963.El 964.Sh SEE ALSO 965.Xr gdb 1 , 966.Xr nm 1 , 967.Xr sysctl.conf 5 , 968.Xr hangman 6 , 969.Xr kgdb 7 , 970.Xr crash 8 , 971.Xr sysctl 8 , 972.Xr extent 9 , 973.Xr pool 9 , 974.Xr uvm 9 975.Sh HISTORY 976This kernel facility first appeared in the MACH 2 operating system 977developed by CMU. 978Hangman (which stands for "hangs maniacal analyzer") first appeared in 979.Ox 1.2 . 980