1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036103710381039104010411042104310441045104610471048104910501051105210531054105510561057105810591060106110621063106410651066106710681069107010711072107310741075107610771078107910801081108210831084108510861087108810891090109110921093109410951096109710981099 |
- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
- <html>
- <head>
- <title>
- CS161 GDB Guide
- </title>
- <link rel="shortcut icon" href="favicon.ico">
- <style type="text/css">
- a.menubarlink:link { color: white; font-weight: bold; }
- a.menubarlink:visited { color: white; font-weight: bold; }
- a.menubarlink:active { color: white; font-weight: bold; }
- a.menubarlink:hover { color: white; font-weight: bold; }
- </style>
- </head>
- <body bgcolor="#ffffff">
- <a name="top"></a>
- <h1 align=center>CS 161: Operating Systems</h1>
- <h2 align=center>Tuesday/Thursday 1:00-2:30</h2>
- <h2 align=center>Pierce 301</h2>
- <hr>
- <table width=100%>
- <tr>
- <td align=center bgcolor="firebrick">
- <a href="../index.html" class="menubarlink">
- Home
- </a>
- </td>
- <td align=center bgcolor="firebrick">
- <a href="../syllabus.html" class="menubarlink">
- Syllabus
- </a>
- </td>
- <td align=center bgcolor="firebrick">
- <a href="../assignments.html" class="menubarlink">
- Assignments
- </a>
- </td>
- <td align=center bgcolor="firebrick">
- <a href="../resources.html" class="menubarlink">
- Resources
- </a>
- </td>
- <td align=center bgcolor="firebrick">
- <a href="http://piazza.com/harvard/spring2016/cs161/home" class="menubarlink">
- Piazza
- </a>
- </td>
- </tr>
- </table>
- <hr>
- <h2>What GDB is and why you need to use it</h2>
- <p>
- GDB is a debugger.
- The fundamental point of a debugger is to stop and inspect the state
- of a running program.
- This helps you analyze the behavior of
- your program, and thus helps diagnose bugs or mistakes.
- With GDB you
- can (in general) do the following things:
- <ul>
- <li>Control aspects of the environment that your program will run in.</li>
- <li>Start your program or connect to an already-running program.</li>
- <li>Make your program stop for inspection.</li>
- <li>Step through your program one line at a time or one machine
- instruction at a time.</li>
- <li>Inspect the state of your program once it has stopped.</li>
- <li>Change the state of your program and then allow it to resume
- execution.</li>
- </ul>
- </p>
- <p>
- In your previous programming experience, you may have managed without
- using a debugger.
- You might have been able to find the mistakes in your
- programs by printing things on the screen or simply reading through your
- code.
- Beware, however, that OS/161 is a large and complex body of code,
- much more so than you may have worked on in the past.
- To make matters
- worse, you didn't write most of it.
- A debugger
- is an essential tool in this environment.
- We would not lie if we said
- that there has never been a student in CS161 who has survived this class
- without using GDB.
- You should, therefore, take the time to learn GDB and
- make it your best friend.
- (Or rather, your second best friend; your best
- friend should be your partner.)
- This guide will explain to you how to
- get started debugging OS/161, describe the most common GDB commands, and
- suggest some helpful debugging techniques.
- </p>
- <h2>How to start debugging</h2>
- <p>
- Because you're trying to debug the OS/161 kernel, which is running
- inside System/161 (and not System/161 itself) you can't just do
- <pre>
- gdb sys161
- </pre>
- as that runs the debugger on System/161.
- Nor can you do
- <pre>
- gdb kernel
- (gdb) run
- </pre>
- as that tries to run the kernel outside of System/161, which isn't
- going to work.
- </p>
- <p>
- Instead, you need to connect the debugger to the kernel via
- System/161's debugger interface.
- Also, you can't use the regular system copy of GDB, as the OS/161
- kernel is MIPS and you need a version of GDB configured to debug MIPS.
- Among the tools for working with OS/161 is a copy of GDB so
- configured; you can run it as <tt>os161-gdb</tt>.
- </p>
- <p>
- The complete procedure is as follows.
- <ol>
- <li>Open two shell windows. These must be logged in on the same
- machine -- in our case both in the CS50 Appliance, but if you're
- working in some other environment you might need to take special steps
- to arrange this.</li>
- <li>In both windows, change to the OS/161 root directory.
- <pre>
- % cd ~/cs161/root
- </pre>
- </li>
- <li>In one window, boot OS/161 in System/161, and use the <tt>-w</tt>
- option to tell System/161 to wait for a debugger connection:
- <pre>
- % sys161 -w kernel
- </pre>
- </li>
- <li>In the other window, start <tt>os161-gdb</tt> on the same kernel
- image, and tell gdb to connect to System/161:
- <pre>
- % os161-gdb kernel
- (gdb) target remote unix:.sockets/gdb
- </pre>
- </li>
- </ol>
- At this point, if everything worked properly, GDB will connect to your
- kernel and
- tell you that the target program is stopped in <tt>start.S</tt>.
- It is, in fact, waiting at the very first instruction of the kernel, as
- if you'd just started the kernel in GDB.
- It also normally can't find the source listing for <tt>start.S</tt>;
- there's a way to fix that (see below) but it doesn't matter just yet.
- </p>
- <p>
- Now you can use GDB to debug almost exactly as you would debug an
- ordinary program.
- Generally you <em>aren't</em> interested in debugging
- <tt>start.S</tt>, so the first thing you typically want to do is set
- one or more breakpoints and then continue the kernel until it reaches
- one.
- </p>
- <h2>A first GDB run</h2>
- <p>
- This first time, you aren't hunting a specific bug, you're just trying
- things out; so a good place to put a breakpoint is
- <tt>kmain</tt>.
- This is the kernel's equivalent of an ordinary C program's
- <tt>main</tt>: the beginning of the program, where execution moves out
- of the preliminary assembly-language startup code into C code.
- Start up the kernel in GDB as above, and then put a breakpoint on
- <tt>kmain</tt> as follows:
- <pre>
- (gdb) break kmain
- </pre>
- Now continue:
- <pre>
- (gdb) c
- </pre>
- and GDB will stop again in <tt>main.c</tt>.
- You'll be stopped on a line that calls <tt>boot()</tt>, the function
- that sequences the kernel's initial bootup actions.
- Single-step into this:
- <pre>
- (gdb) s
- </pre>
- This will take you to the first line of <tt>boot</tt>, which is a call
- to <tt>kprintf</tt>, the OS/161 function for printing miscellaneous
- messages to the system console.
- Single-step over this call:
- <pre>
- (gdb) n
- </pre>
- to execute the <tt>kprintf</tt> call and stop on the next line.
- You can get the listing of the area around where you are by typing
- <pre>
- (gdb) list
- </pre>
- You'll notice that even though there are several calls to
- <tt>kprintf</tt> here, nothing actually comes out on the console.
- Single-step with <tt>n</tt> over the next few lines (first some
- <tt>kprintf</tt> calls, then calls for bootstrapping various kernel
- subsystems) until you get to <tt>mainbus_bootstrap</tt>.
- </p>
- <p>
- When you step over <tt>mainbus_bootstrap</tt>, suddenly all the
- console messages print out.
- This is because nothing can actually come out of the console until the
- kernel searches for and finds the console hardware; this happens
- inside <tt>mainbus_bootstrap</tt>.
- This is important to remember: if the kernel hangs or dies without
- printing anything, it does not mean that something inexplicably
- horrible happened before that first <tt>kprintf</tt>, it just means
- that something happened before the call to <tt>mainbus_bootstrap</tt>.
- You <em>will</em> be writing code that happens before there, so
- chances are this will happen to you at least once this semester.
- One of the reasons GDB is important: tracking such problems down with
- GDB is pretty easy.
- Tracking them down without GDB, when you can't print anything out, is
- ... interesting.
- </p>
- <p>
- Now tell GDB to execute through to the end of <tt>boot</tt>:
- <pre>
- (gdb) finish
- </pre>
- It will come to a stop again on the second line of <tt>kmain</tt>,
- which calls the kernel menu code.
- You can step into this to see how the menu works, but it's actually
- not all that interesting.
- So instead, put a breakpoint on the <tt>reboot</tt> system call
- handler and continue:
- <pre>
- (gdb) break sys_reboot
- (gdb) c
- </pre>
- Now the kernel will print the menu prompt waiting for input.
- Run the <tt>poweroff</tt> utility, by typing (in the other window):
- <pre>
- OS/161 kernel [? for menu]: p /sbin/poweroff
- </pre>
- If you step through this you'll see that after attending to various
- shutdown tasks, it calls <tt>mainbus_poweroff</tt> to shut off the
- electricity on the system board.
- At that point System/161 will exit, and GDB will print
- <tt>
- Remote connection closed
- </tt>
- and you're done.
- <h2>Connecting and disconnecting GDB</h2>
- <p>
- While you often know that you want to run in the debugger when starting the
- kernel, in which case you use the method above, this isn't always the
- case.
- You can attach GDB to System/161 at any time by simply running it and
- entering the <tt>target remote</tt> line.
- This connects to System/161's debugger port, and that causes
- System/161 to stop.
- </p>
- <p>
- System/161 will itself also sometimes stop and wait for a debugger
- connection.
- This will happen on various hardware-level conditions that are not
- normally supposed to happen; it will also happen if the kernel
- executes a breakpoint instruction or requests the debugger explicitly
- via the <tt>ltrace</tt> device.
- And it will happen if you turn System/161's progress monitoring on
- (see the System/161 manual for an explanation) and the kernel fails to
- make progress in the specified amount of time.
- In these cases you attach the debugger in the same way: by running it
- and entering the <tt>target remote</tt> line.
- </p>
- <p>
- Importantly, you can also make System/161 stop and wait for the
- debugger by pressing ^G.
- This can be
- useful if your kernel is looping or deadlocked.
- </p>
- <p>
- Because typing that <tt>target remote</tt> command over and over is a
- major nuisance, we suggest setting up a macro for it.
- Edit the file <tt>~/cs161/root/.gdbinit</tt> and put the following in
- it:
- <pre>
- define db
- target remote unix:.sockets/gdb
- end
- </pre>
- On startup, GDB reads the file <tt>.gdbinit</tt> in the current
- directory and executes the GDB commands it finds in it; so now you can
- connect to System/161 by just typing
- <pre>
- (gdb) db
- </pre>
- It is also often useful to put common breakpoint settings in
- <tt>.gdbinit</tt>, such as on <tt>panic</tt> or <tt>badassert</tt>.
- (Note that as of OS/161 2.0, <tt>panic</tt> will itself automatically
- request a debugger connection.
- But you may find that putting an explicit breakpoint on it is still
- helpful.)
- </p>
- <p>
- Note that for reasons we don't yet understand (if we understood them,
- we have have fixed this),
- sometimes connecting the debugger while System/161 is running
- (rather than waiting for a debugger connection) doesn't work.
- Also sometimes, but not usually the same times, pressing ^G doesn't
- seem to work.
- (If you manage to get a repeatable test case for either of these
- conditions, contact dholland@eecs.harvard.edu.)
- </p>
- <p>
- Also note that if GDB is already connected, and you tell it to
- continue and the kernel doesn't stop again when you expected, getting
- it to stop so you can do more with GDB can be annoyingly difficult.
- Theoretically at this point pressing ^G in the System/161 window will
- cause it to stop executing and return control to GDB, but sometimes
- this doesn't work either.
- Hitting ^C in GDB will get you a GDB prompt again, but one where you
- can't do much of anything because the kernel is still running; GDB
- suggests you type <tt>interrupt</tt>, but this doesn't work.
- If all else fails, you can kill GDB, start another copy, and
- reconnect.
- Don't hit ^C in System/161 as that will kill it and lose your state.
- </p>
- <p>
- When you are done
- debugging, you can disconnect the debugger from System/161 (and thus
- the running kernel) in two ways:
- <pre>
- (gdb) detach
- </pre>
- will unhook the debugger and leave the kernel still running, whereas
- <pre>
- (gdb) kill
- </pre>
- will unceremoniously nuke System/161, much as if you'd gone to its
- window and typed ^C.
- Quitting out of gdb while connected is the same as using <tt>kill</tt>.
- </p>
- <p>
- Caution: Be sure that the kernel image that you run in System/161 and
- the one
- you run <tt>os161-gdb</tt> on are the same.
- If they aren't, it's like using a different edition of a textbook with
- different page numbers and not realizing it: things can get very
- bizarre and very confusing.
- </p>
- <h2>Executing under GDB control</h2>
- <p>
- Here are the most common and often-used commands for controlling the
- execution of the kernel in GDB:
- </p>
- <p>
- <b>s, step</b> - Step through the program.
- If you want to go through your program step by step after it has hit
- a breakpoint, use the "step" command.
- Typing
- <pre>
- (gdb) s
- </pre>
- will execute the next line of code.
- If the next line is a function call,
- the debugger will step into this function.
- </p>
- <p>
- <b>n, next</b> - Execute the next line.
- This command is similar to the "step" command, except that it does not
- step into a function, but executes the function, as if it were a simple
- statement. This is extraordinarily useful for stepping over functions
- such as kprintf or other functions that you know/believe to be correct.
- </p>
- <p>
- <b>adv, advance</b> - Advance to a specified line
- This is like "next" but continues to a specified line number.
- It is like setting a breakpoint on that line and using "continue".
- If you accidentally pick a line that you won't get to, it won't stop
- until the next breakpoint.
- </p>
- <p>
- <b>finish</b> - Continue until function return.
- This command advances until the end of the function.
- </p>
- <p><b>c, continue</b> - continue execution.
- When the kernel is stopped (at a breakpoint, or after connecting, or
- after executing manually for a while), type
- <pre>
- (gdb) c
- </pre>
- to just continue executing.
- Execution continues until something happens that stops it, such as
- hitting a breakpoint, or if you type ^G into System/161.
- (See above for some notes on ^G.)
- </p>
- <p>
- <b>b, break</b> - set a breakpoint.
- Use this command to specify that your program should stop execution at
- a certain line or function.
- Typing
- <pre>
- (gdb) b 18
- </pre>
- means that your program will stop every time it executes a statement
- on line 18.
- As with the "list" command, you can specify break on a function, e.g.:
- <pre>
- (gdb) b main
- </pre>
- will stop when execution encounters the main function.
- </p>
- <p>
- <b>info breakpoints</b> - List all breakpoints.
- </p>
- <p>
- <b>d, delete</b> - Delete a breakpoint by number.
- Use this command to delete a breakpoint.
- By typing
- <pre>
- (gdb) d 1
- </pre>
- you will delete the breakpoint numbered 1.
- GDB displays the number of
- a breakpoint when you set that breakpoint.
- Typing "d" without arguments
- deletes all breakpoints.
- </p>
- <p><b>clear</b> - Clear a breakpoint by name/address.
- If you don't remember the number of the breakpoint you want to delete,
- use the "clear" command.
- Just like the "breakpoint" command, it takes
- a line number or a function name as an argument.
- </p>
- <p>
- <b>disable, enable</b> - Disable/enable a breakpoint.
- Use these commands with a breakpoint number as an argument to disable
- or enable a breakpoint.
- When you disable a breakpoint, the breakpoint is still there, but execution
- will not stop at it until you enable the breakpoint.
- </p>
- <p>
- <b>cond</b> - Make a breakpoint conditional.
- This can be helpful if you need to set a breakpoint on a very common
- code path.
- Usage is like this:
- <pre>
- (gdb) cond 3 count > 1000
- </pre>
- which makes breakpoint 3 only happen when the expression
- <tt>count > 1000</tt> is true.
- CAUTION: in past years, we have found that use of conditional
- breakpoints seems to corrupt GDB's internal state such that it starts
- lying to you.
- This year (2016) we have a substantially newer GDB version that
- we hope no longer has this bug.
- Until we have further data, use conditional breakpoints cautiously,
- and let the course staff know what your experiences are.
- </p>
- <p>
- <b>command</b> - Execute a command on a breakpoint.
- You can specify that a
- certain command or a set of commands be executed at a breakpoint.
- For
- example, to specify that a certain string and a certain value are printed
- every time you stop at breakpoint 2, you could type:
- <pre>
- (gdb) command 2
- > printf "theString = %s\n", theString
- > print /x x
- > end
- </pre>
- </p>
- <h2>Inspecting stuff with GDB</h2>
- <p>
- The chief purpose of a debugger is to inspect the target program.
- Here are the most common/useful commands for doing that.
- </p>
- <p>
- <b>l, list</b> - List lines from source files.
- Use this command to display
- parts of your source file.
- For example, typing
- <pre>
- (gdb) l file.c:101
- </pre>
- will display line 101 in <tt>file.c</tt>.
- If you leave off the file name, GDB will use what it thinks is the
- current source file; this is usually the most recently referenced
- source file but occasionally isn't what one expects.
- You can also give a function name to list starting at the top of that
- function.
- And if you give no arguments GDB will list the next chunk of the file
- you last listed or if you just stopped execution, the point at which you
- stopped.
- </p>
- <p>
- Note: If you need to debug in <tt>start.S</tt> (or any other assembler
- file) GDB can't find the sources by default.
- This is because of a glitch in the toolchain: the paths to the
- assembler source files are relative, not absolute, so they only work
- by default in the kernel build directory.
- To work around it, you can tell GDB to start looking for the sources
- in the kernel build directory:
- <pre>
- (gdb) directory /home/jharvard/cs161/os161/kern/compile/ASSTN
- </pre>
- for whatever assignment <tt>N</tt> you're currently working on.
- Tip: you can put this in <tt>.gdbinit</tt>.
- </p>
- <p>
- <b>disassemble</b> - Disassemble some code.
- Use this command to print disassembly of the machine code for a
- function.
- Add <tt>/m</tt> to mix in the source listing as well.
- Hopefully you won't need to use this; but it can occasionally be
- useful.
- </p>
- <p>
- <b>x</b> - Examine memory.
- This is a general-purpose function for dumping memory; it has a
- variety of options and forms, all of which take a memory address
- (which can be the name of a variable or function) as an argument.
- The most useful are:
- <ul>
- <li><tt>x/x</tt> - dump memory as hex</li>
- <li><tt>x/4x</tt> - dump memory as hex four at a time</li>
- <li><tt>x/64x</tt> - dump memory as hex 64 at a time, which fits 256
- bytes nicely on a normal-size screen</li>
- <li><tt>x/s</tt> - dump memory as null-terminated strings</li>
- <li><tt>x/c</tt> - dump memory as single characters</li>
- <li><tt>x/i</tt> - dump memory as disassembly</li>
- </ul>
- but there are many more variations; see the GDB help on the command
- for the full details.
- </p>
- <p>
- <b>bt, where</b> - Display backtrace.
- To find out where you are in the execution of your program and how you
- got there, use one of these commands.
- This will show the backtrace of
- the execution, including the function names and arguments.
- </p>
- <p>
- <b>up</b>, <b>down</b> - Navigate the backtrace.
- You can move up the stack to parent stack frames to inspect things
- there.
- With both the "up" and "down" commands you can give an optional
- numeric argument to move up or down that many frames instead of just
- one.
- </p>
- <p>
- <b>frame</b> - Select a stack frame in the backtrace.
- With a numeric argument, this moves up (or down, as needed) the stack
- trace to the frame with the specified number.
- (Frame 0 is the innermost, where execution is actually occurring.)
- Without an argument it reselects the last frame you navigated to,
- which both prints it out and has the often-useful side effect of
- resetting the current listing position to there.
- </p>
- <p>
- <b>p, print</b> - Print an expression.
- Use for example
- <pre>
- (gdb) p name
- </pre>
- to print the value of <tt>name</tt>.
- You can use arbitrarily complex C expressions; it is often useful for
- example to include typecasts.
- Normally the value will be printed according to its type; you can
- insert insert <tt>/x</tt>, <tt>/s</tt>, <tt>/d</tt>, <tt>/u</tt>
- to force printing as hex, as a string, or as a signed or unsigned
- integer respectively.
- </p>
- <p>
- When you print something, you'll get a result line like this:
- <pre>
- $1 = 0x80063d80 ""
- </pre>
- The <tt>$1</tt> is a <em>convenience variable</em>, a scratch variable
- created and remembered by GDB for your subsequent use.
- You can use <tt>$1</tt> as a value in subsequent expressions and it
- will retrieve the value that was printed; this saves having to retype
- it or cut and paste it.
- (Note though that it saves the <em>value</em>, not the
- <em>expression</em>; if you execute for a while and then use
- <tt>$1</tt> again it will have the same value regardless of what's
- happened in the meantime to the values in the expression that
- generated it.
- </p>
- <p>
- <b>printf</b> - Formatted print.
- The "printf" command allows you to specify the formatting of the output,
- just like you do with a C library printf() function.
- For example, you
- can type:
- <pre>
- (gdb) printf "X = %d, Y = %d\n",X,Y
- </pre>
- </p>
- <p>
- <b>info registers</b> - Show the processor registers.
- This prints all the processor registers.
- You can also use values from registers in "print" expressions:
- a <tt>$</tt> followed by the symbolic name of the register fetches
- that register's value; e.g. the return value of a function is normally
- left in <tt>$v0</tt>.
- </p>
- <p>
- <b>display</b> - Display an expression.
- Compute and display the value of an expression every time the program
- stops.
- For example:
- <pre>
- (gdb) display abc
- </pre>
- If there is no <tt>abc</tt> in scope when the program stops, the
- display won't appear.
- This command is otherwise the same as "print", including the support
- for type modifiers.
- </p>
- <p>
- <b>undisp</b> - Remove a "display".
- The argument is the number of the "display" expression to remove.
- With no argument, all "display" expressions are removed.
- Also, <tt>delete display N</tt> is the same as <tt>undisp N</tt>.
- </p>
- <p>
- <b>set variable</b> - Assign a value to a variable.
- Sometimes it is useful to change the value of a variable while the
- program is running.
- For example if you have a variable "x", typing
- <pre>
- (gdb) set variable x = 15
- </pre>
- will change the value of "x" to 15.
- </p>
-
- <h2>Other GDB commands to note</h2>
- <p>
- <b>define</b> - Define a new command.
- Use this if you want to define a new command.
- This is similar to assigning
- a macro and can save you typing.
- Just as with a macro, you can put
- together several commands.
- An example of defining a short form of the <tt>target remote</tt>
- command for attaching to System/161 was provided above.
- Some other more elaborate examples can be found in the
- <tt>gdbscripts</tt> directory in the OS/161 source tree.
- </p>
- <p>
- <b>help</b> - Get help (on gdb, not your CS161 assignments!)
- To find more about a particular command just type:
- <pre>
- (gdb) help <command name>
- </pre>
- Note that several of the commands listed above (e.g. "info
- breakpoints") are specific instances of more general GDB commands,
- like "info" and "set".
- </p>
- <h2>GDB threads support</h2>
- <p>
- Nowadays GDB understands the concept of multiple threads; you can
- switch from one thread to another while debugging in order to inspect all the
- available execution contexts.
- The good news is: this now works with System/161.
- The not so good news is: GDB threads map to CPUs, not to OS/161 kernel
- threads.
- This means that while you can inspect all the CPUs that are executing,
- you still can't easily inspect sleeping threads.
- Unfortunately, GDB's thread model combined with the fact that the
- debugger talks transparently to System/161 makes this mapping more or
- less the only choice.
- </p>
- <p>
- When stopped in GDB, list the threads like this:
- <pre>
- (gdb) info threads
- </pre>
- This will give a display something like this:
- <pre>
- Id Target Id Frame
- 4 Thread 13 (CPU 3) wait () at ../../arch/mips/thread/cpu.c:279
- 3 Thread 12 (CPU 2) wait () at ../../arch/mips/thread/cpu.c:279
- 2 Thread 11 (CPU 1) wait () at ../../arch/mips/thread/cpu.c:279
- * 1 Thread 10 (CPU 0) runprogram (progname=0x800f3f40 "/bin/sh")
- at ../../syscall/runprogram.c:492
- </pre>
- The leftmost column is the thread number that you need to use while
- talking to GDB, and the CPU number listed is the System/161 CPU
- number.
- Unfortunately the GDB thread numbers are offset by 1 at the start and
- may diverge further over time at GDB's whim.
- (The "Thread 13" number is the number used in the communications
- channel between GDB and System/161; you don't need to care about it.
- It's offset from the CPU number by 10 because if you use 0 GDB dumps
- core.)
- </p>
- <p>
- To switch to another CPU, use the "thread" command with the number
- from the leftmost "Id" column:
- <pre>
- (gdb) thread 1
- </pre>
- </p>
- <h2>The GDB "text user interface"</h2>
- <p>
- GDB has a curses-based "text user interface" (TUI) that gives you a
- slightly less 1980 experience when debugging.
- This gives you a split-screen window with the current source file in
- the top part and the GDB prompt in the bottom part.
- Type <tt>^X a</tt> (that's control-X followed by the letter a)
- after starting GDB to
- enable it, and type that again to make it go away.
- </p>
- <p>
- In TUI mode most GDB commands work as before; there are just a
- couple of keystrokes you need to know about:
- <ul>
- <li>^L - redraw screen (like in many programs)</li>
- <li>^X o - switch to other panel in the GDB window (like in Emacs)</li>
- </ul>
- By default the current panel is the source listing, but typing goes
- into the GDB prompt panel anyway; however, to use the arrow keys for
- input editing you need to switch to the GDB prompt panel.
- </p>
- <p>
- There is also a graphical front end to GDB called DDD; it should be
- possible to use this with the OS/161 version of GDB.
- No current information on this is available right now, unfortunately.
- </p>
- <h2>Debugging tips</h2>
- <p>
- Tip #1: Check your beliefs about the program
- </p>
- <p>
- So how do you actually approach debugging? When you have a bug in a
- program, it means that you have a particular belief about how your program
- should behave, and somewhere in the program this belief is violated.
- For
- example, you may believe that a certain variable should always be 0 when
- you start a "for" loop, or a particular pointer can never be NULL in
- a certain "if statement".
- To check such beliefs, set a breakpoint in
- the debugger at a line where you can check the validity of your belief.
- And when your program hits the breakpoint, ask the debugger to display
- the value of the variable in question.
- </p>
- <p>
- Tip #2: Narrow down your search
- </p>
- <p>
- If you have a situation where a variable does not have the value you
- expect, and you want to find a place where it is modified, instead of
- walking through the entire program line by line, you can check the value
- of the variable at several points in the program and narrow down the
- location of the misbehaving code.
- </p>
- <p>
- Tip #3: Walk through your code
- </p>
- <p>
- Steve Maguire (the author of Writing Solid Code) recommends using
- the debugger to step through every new line of code you write, at
- least once, in order to understand exactly what your code is doing.
- It helps you visually verify that your program is behaving more or less
- as intended.
- With judicious use, the step, next and finish commands
- can help you trace through complex code quickly and make it possible to
- examine key data structures as they are built.
- </p>
- <p>
- Tip #4: Use good tools
- </p>
- <p>
- Using GDB with a visual front-end can be very helpful.
- For example,
- using GDB inside the emacs editor puts you in a split-window mode,
- where in one of the windows you run your GDB session, and in the other
- window the GDB moves an arrow through the lines of your source file as
- they are executed.
- To use GDB through emacs do the following:
- <ol>
- <li> Start emacs.
- <li> Type the "meta" key followed by an "x".
- <li> At the prompt type "gdb". Emacs will display the message:
- <pre>
- Run gdb (like this): gdb
- </pre>
- <li> Delete the word "gdb", and type:
- <pre>
- os161-gdb kernel
- </pre>
- So in the end you should have:
- <pre>
- Run gdb (like this): os161-gdb kernel
- </pre>
- displayed in the control window.
- </ol>
- </p>
- <p>
- At this point you can continue using GDB as explained in section 2.
- </p>
- <p>
- Tip #5: Beware of printfs!
- </p>
- <p>
- A lot of programmers like to find mistakes in their programs by inserting
- "printf" statements that display the values of the variables.
- If you
- decide to resort to this technique, you have to keep in mind two things:
- First, because adding printfs requires a recompile, printf debugging
- may take longer overall than using a debugger.
- </p>
- <p>
- More subtly, if you are debugging a multi-threaded program, such as
- a kernel, the order in which the instructions are executed depends on
- how your threads are scheduled, and some bugs may or may not manifest
- themselves under a particular execution scenario.
- Because printf outputs
- to the console, and the console in System/161 is a serial device that
- isn't extraordinarily fast, an extra call to printf may alter the timing
- and scheduling considerably.
- This can make bugs hide or appear to come
- and go, which makes your debugging job much more difficult.
- </p>
- <p>
- To help address this problem, System/161 provides a simple debug output
- facility as part of its trace control device.
- One of the trace control
- device's registers, when written to, prints a notice in the System/161
- output including the value that was written.
- In OS/161, provided your
- System/161 has been configured to include the trace control device, you
- can access this feature by calling trace_debug(), which is defined in
- dev/lamebus/ltrace.h.
- While this is less disruptive than calling printf,
- it is still not instant and can still alter the timing of execution.
- By
- contrast, the System/161 debugger interface is completely invisible; as
- far as your kernel is concerned, time is stopped while you are working
- in the debugger.
- </p>
- <p>
- Tip #6: Debugging deadlocks
- </p>
- <p>
- A deadlock occurs when two or more threads are all waiting for the
- others to proceed.
- (We'll talk more about this in lecture.)
- In a simple deadlock there'll be one thread T1 that already holds a lock L1,
- and another thread T2 that already holds a lock L2.
- T1 is waiting to get L2, so it can't proceed until T2 runs, and T2 is
- waiting to get L1, so it can't proceed until T1 runs.
- The goal of debugging a deadlock is to find out the identities of T1
- and T2, and of L1 and L2, and then figure out what combination of
- circumstances and code paths allowed them to get into this state.
- </p>
- <p>
- To do this, you generally want to begin by finding one of the threads,
- since there are usually a lot more locks in the system than threads.
- Each thread that's blocked reports what it's blocked on in the field
- <tt>t_wchan_name</tt>.
- The basic procedure is to locate a thread (the first ones to start
- with are the ones currently on the CPUs; after that look in your
- process table that you wrote for assignment 2), check what it's
- waiting for, then find who's holding that.
- If it's waiting for a lock, hopefully the locks you wrote record who's
- holding them.
- That gives you another thread to examine; repeat until you find a
- loop.
- If it's waiting on a CV, it gets a bit more interesting because you
- need to figure out who was supposed to signal that CV and didn't... or
- if they did, why the waiting thread didn't receive the signal.
- For deadlocks involving spinlocks, remember that in OS/161 spinlocks
- are held by CPUs, not threads.
- </p>
- <p>
- Once you have the participants identified, the stack traces from those
- threads will usually (but, alas, not always) give you enough
- information to work out what happened.
- </p>
- <p>
- It is often helpful to set up a global table of all locks and/or CVs
- (OS/161 already comes with a global table of all wchans) to make this
- process easier.
- It is also possible, and potentially worthwhile, to hack more state
- information into the thread structure.
- One can even write an automatic deadlock detector, although this
- probably has a bad cost/benefit ratio and won't help you much when CVs
- are involved.
- </p>
- <p>
- Tip #7: Debugging assembly
- </p>
- <p>
- When GDB stops in an assembly source file (a .S file) various special
- considerations apply.
- GDB is meant to be a source-level debugger for high
- level languages and isn't very good at debugging assembly.
- So various
- tricks are required to get adequate results.
- </p>
- <p>
- The OS/161 toolchain tells the assembler to emit line number
- information for assembly files, but as noted above (under "list") you
- need to tell GDB how to find them.
- Do that.
- Then you can at least see the file you're working on.
- </p>
- <p>
- It is also sometimes helpful to disassemble the kernel; type
- <pre>
- % os161-objdump -d kernel | less
- </pre>
- in another window and page or search through it as needed.
- </p>
- <p>
- To single step through assembler, use the nexti and stepi commands,
- which are like next and step but move by one instruction at a time.
- </p>
- <p>
- The command x /i (examine as instructions) is useful for disassembling
- regions from inside GDB.
- </p>
- <p>
- Print register values with <tt>$</tt> and the symbolic register names
- as described above (<tt>$v0</tt>, <tt>$a0</tt>, etc.) to see the
- values that are being
- handled.
- </p>
- <p>
- Tip #8: trace161
- </p>
- <p>
- The trace161 program is the same as sys161 but includes additional
- support for various kinds of tracing and debugging operations.
- You can
- have System/161 report disk I/O, interrupts, exceptions, or whatever.
- See
- the System/161 documentation for more information.
- </p>
- <p>
- One of the perhaps more interesting trace options is to have System/161
- report every machine instruction that is executed, either at user level,
- at kernel level, or both.
- Because this setting generates vast volumes
- of output, it's generally not a good idea to turn it on from the command
- line.
- (It is sometimes useful, however, in the early stages of debugging
- assignment 2 or 3, to log all user-mode instructions.)
- However, the
- trace options can be turned on and off under software control using
- the System/161 trace control device.
- It can be extremely useful to turn
- instruction logging on for short intervals in places you suspect something
- strange is happening.
- See dev/lamebus/ltrace.h for further information.
- </p>
- <p>
- Tip #9: casting void pointers
- </p>
- <p>
- If you have a void * in GDB and you know what type it actually is, you
- can cast it when printing, using the usual C expression syntax.
- This is very helpful when dealing with arrays, for example.
- </p>
- <p>
- Tip #10: tracing through exceptions
- </p>
- <p>
- When you get a stack backtrace and it reaches an exception frame, GDB can
- sometimes now trace through the exception frame, but it doesn't always
- work very well.
- Sometimes it only gets one function past the exception,
- and sometimes it skips one function.
- (This is a result of properties of
- the MIPS architecture and the way GDB is implemented and doesn't appear
- readily fixable.)
- Always check the tf_epc field of the trap frame to
- see exactly where the exception happened, and if in doubt, cross-check
- it against a disassembly or have GDB disassemble the address.
- </p>
- <h2>Where to go for help</h2>
- <p>
- For help on GDB commands, type "help" from inside GDB.
- You can find the
- <a href="http://www.gnu.org/software/gdb/gdb.html">GDB manual here</a>
- and of course your friendly TFs are always there to help!
- </p>
- </body>
- </html>
|