gdb161.html 34 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036103710381039104010411042104310441045104610471048104910501051105210531054105510561057105810591060106110621063106410651066106710681069107010711072107310741075107610771078107910801081108210831084108510861087108810891090109110921093109410951096109710981099
  1. <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
  2. <html>
  3. <head>
  4. <title>
  5. CS161 GDB Guide
  6. </title>
  7. <link rel="shortcut icon" href="favicon.ico">
  8. <style type="text/css">
  9. a.menubarlink:link { color: white; font-weight: bold; }
  10. a.menubarlink:visited { color: white; font-weight: bold; }
  11. a.menubarlink:active { color: white; font-weight: bold; }
  12. a.menubarlink:hover { color: white; font-weight: bold; }
  13. </style>
  14. </head>
  15. <body bgcolor="#ffffff">
  16. <a name="top"></a>
  17. <h1 align=center>CS 161: Operating Systems</h1>
  18. <h2 align=center>Tuesday/Thursday 1:00-2:30</h2>
  19. <h2 align=center>Pierce 301</h2>
  20. <hr>
  21. <table width=100%>
  22. <tr>
  23. <td align=center bgcolor="firebrick">
  24. <a href="../index.html" class="menubarlink">
  25. Home
  26. </a>
  27. </td>
  28. <td align=center bgcolor="firebrick">
  29. <a href="../syllabus.html" class="menubarlink">
  30. Syllabus
  31. </a>
  32. </td>
  33. <td align=center bgcolor="firebrick">
  34. <a href="../assignments.html" class="menubarlink">
  35. Assignments
  36. </a>
  37. </td>
  38. <td align=center bgcolor="firebrick">
  39. <a href="../resources.html" class="menubarlink">
  40. Resources
  41. </a>
  42. </td>
  43. <td align=center bgcolor="firebrick">
  44. <a href="http://piazza.com/harvard/spring2016/cs161/home" class="menubarlink">
  45. Piazza
  46. </a>
  47. </td>
  48. </tr>
  49. </table>
  50. <hr>
  51. <h2>What GDB is and why you need to use it</h2>
  52. <p>
  53. GDB is a debugger.
  54. The fundamental point of a debugger is to stop and inspect the state
  55. of a running program.
  56. This helps you analyze the behavior of
  57. your program, and thus helps diagnose bugs or mistakes.
  58. With GDB you
  59. can (in general) do the following things:
  60. <ul>
  61. <li>Control aspects of the environment that your program will run in.</li>
  62. <li>Start your program or connect to an already-running program.</li>
  63. <li>Make your program stop for inspection.</li>
  64. <li>Step through your program one line at a time or one machine
  65. instruction at a time.</li>
  66. <li>Inspect the state of your program once it has stopped.</li>
  67. <li>Change the state of your program and then allow it to resume
  68. execution.</li>
  69. </ul>
  70. </p>
  71. <p>
  72. In your previous programming experience, you may have managed without
  73. using a debugger.
  74. You might have been able to find the mistakes in your
  75. programs by printing things on the screen or simply reading through your
  76. code.
  77. Beware, however, that OS/161 is a large and complex body of code,
  78. much more so than you may have worked on in the past.
  79. To make matters
  80. worse, you didn't write most of it.
  81. A debugger
  82. is an essential tool in this environment.
  83. We would not lie if we said
  84. that there has never been a student in CS161 who has survived this class
  85. without using GDB.
  86. You should, therefore, take the time to learn GDB and
  87. make it your best friend.
  88. (Or rather, your second best friend; your best
  89. friend should be your partner.)
  90. This guide will explain to you how to
  91. get started debugging OS/161, describe the most common GDB commands, and
  92. suggest some helpful debugging techniques.
  93. </p>
  94. <h2>How to start debugging</h2>
  95. <p>
  96. Because you're trying to debug the OS/161 kernel, which is running
  97. inside System/161 (and not System/161 itself) you can't just do
  98. <pre>
  99. gdb sys161
  100. </pre>
  101. as that runs the debugger on System/161.
  102. Nor can you do
  103. <pre>
  104. gdb kernel
  105. (gdb) run
  106. </pre>
  107. as that tries to run the kernel outside of System/161, which isn't
  108. going to work.
  109. </p>
  110. <p>
  111. Instead, you need to connect the debugger to the kernel via
  112. System/161's debugger interface.
  113. Also, you can't use the regular system copy of GDB, as the OS/161
  114. kernel is MIPS and you need a version of GDB configured to debug MIPS.
  115. Among the tools for working with OS/161 is a copy of GDB so
  116. configured; you can run it as <tt>os161-gdb</tt>.
  117. </p>
  118. <p>
  119. The complete procedure is as follows.
  120. <ol>
  121. <li>Open two shell windows. These must be logged in on the same
  122. machine -- in our case both in the CS50 Appliance, but if you're
  123. working in some other environment you might need to take special steps
  124. to arrange this.</li>
  125. <li>In both windows, change to the OS/161 root directory.
  126. <pre>
  127. % cd ~/cs161/root
  128. </pre>
  129. </li>
  130. <li>In one window, boot OS/161 in System/161, and use the <tt>-w</tt>
  131. option to tell System/161 to wait for a debugger connection:
  132. <pre>
  133. % sys161 -w kernel
  134. </pre>
  135. </li>
  136. <li>In the other window, start <tt>os161-gdb</tt> on the same kernel
  137. image, and tell gdb to connect to System/161:
  138. <pre>
  139. % os161-gdb kernel
  140. (gdb) target remote unix:.sockets/gdb
  141. </pre>
  142. </li>
  143. </ol>
  144. At this point, if everything worked properly, GDB will connect to your
  145. kernel and
  146. tell you that the target program is stopped in <tt>start.S</tt>.
  147. It is, in fact, waiting at the very first instruction of the kernel, as
  148. if you'd just started the kernel in GDB.
  149. It also normally can't find the source listing for <tt>start.S</tt>;
  150. there's a way to fix that (see below) but it doesn't matter just yet.
  151. </p>
  152. <p>
  153. Now you can use GDB to debug almost exactly as you would debug an
  154. ordinary program.
  155. Generally you <em>aren't</em> interested in debugging
  156. <tt>start.S</tt>, so the first thing you typically want to do is set
  157. one or more breakpoints and then continue the kernel until it reaches
  158. one.
  159. </p>
  160. <h2>A first GDB run</h2>
  161. <p>
  162. This first time, you aren't hunting a specific bug, you're just trying
  163. things out; so a good place to put a breakpoint is
  164. <tt>kmain</tt>.
  165. This is the kernel's equivalent of an ordinary C program's
  166. <tt>main</tt>: the beginning of the program, where execution moves out
  167. of the preliminary assembly-language startup code into C code.
  168. Start up the kernel in GDB as above, and then put a breakpoint on
  169. <tt>kmain</tt> as follows:
  170. <pre>
  171. (gdb) break kmain
  172. </pre>
  173. Now continue:
  174. <pre>
  175. (gdb) c
  176. </pre>
  177. and GDB will stop again in <tt>main.c</tt>.
  178. You'll be stopped on a line that calls <tt>boot()</tt>, the function
  179. that sequences the kernel's initial bootup actions.
  180. Single-step into this:
  181. <pre>
  182. (gdb) s
  183. </pre>
  184. This will take you to the first line of <tt>boot</tt>, which is a call
  185. to <tt>kprintf</tt>, the OS/161 function for printing miscellaneous
  186. messages to the system console.
  187. Single-step over this call:
  188. <pre>
  189. (gdb) n
  190. </pre>
  191. to execute the <tt>kprintf</tt> call and stop on the next line.
  192. You can get the listing of the area around where you are by typing
  193. <pre>
  194. (gdb) list
  195. </pre>
  196. You'll notice that even though there are several calls to
  197. <tt>kprintf</tt> here, nothing actually comes out on the console.
  198. Single-step with <tt>n</tt> over the next few lines (first some
  199. <tt>kprintf</tt> calls, then calls for bootstrapping various kernel
  200. subsystems) until you get to <tt>mainbus_bootstrap</tt>.
  201. </p>
  202. <p>
  203. When you step over <tt>mainbus_bootstrap</tt>, suddenly all the
  204. console messages print out.
  205. This is because nothing can actually come out of the console until the
  206. kernel searches for and finds the console hardware; this happens
  207. inside <tt>mainbus_bootstrap</tt>.
  208. This is important to remember: if the kernel hangs or dies without
  209. printing anything, it does not mean that something inexplicably
  210. horrible happened before that first <tt>kprintf</tt>, it just means
  211. that something happened before the call to <tt>mainbus_bootstrap</tt>.
  212. You <em>will</em> be writing code that happens before there, so
  213. chances are this will happen to you at least once this semester.
  214. One of the reasons GDB is important: tracking such problems down with
  215. GDB is pretty easy.
  216. Tracking them down without GDB, when you can't print anything out, is
  217. ... interesting.
  218. </p>
  219. <p>
  220. Now tell GDB to execute through to the end of <tt>boot</tt>:
  221. <pre>
  222. (gdb) finish
  223. </pre>
  224. It will come to a stop again on the second line of <tt>kmain</tt>,
  225. which calls the kernel menu code.
  226. You can step into this to see how the menu works, but it's actually
  227. not all that interesting.
  228. So instead, put a breakpoint on the <tt>reboot</tt> system call
  229. handler and continue:
  230. <pre>
  231. (gdb) break sys_reboot
  232. (gdb) c
  233. </pre>
  234. Now the kernel will print the menu prompt waiting for input.
  235. Run the <tt>poweroff</tt> utility, by typing (in the other window):
  236. <pre>
  237. OS/161 kernel [? for menu]: p /sbin/poweroff
  238. </pre>
  239. If you step through this you'll see that after attending to various
  240. shutdown tasks, it calls <tt>mainbus_poweroff</tt> to shut off the
  241. electricity on the system board.
  242. At that point System/161 will exit, and GDB will print
  243. <tt>
  244. Remote connection closed
  245. </tt>
  246. and you're done.
  247. <h2>Connecting and disconnecting GDB</h2>
  248. <p>
  249. While you often know that you want to run in the debugger when starting the
  250. kernel, in which case you use the method above, this isn't always the
  251. case.
  252. You can attach GDB to System/161 at any time by simply running it and
  253. entering the <tt>target remote</tt> line.
  254. This connects to System/161's debugger port, and that causes
  255. System/161 to stop.
  256. </p>
  257. <p>
  258. System/161 will itself also sometimes stop and wait for a debugger
  259. connection.
  260. This will happen on various hardware-level conditions that are not
  261. normally supposed to happen; it will also happen if the kernel
  262. executes a breakpoint instruction or requests the debugger explicitly
  263. via the <tt>ltrace</tt> device.
  264. And it will happen if you turn System/161's progress monitoring on
  265. (see the System/161 manual for an explanation) and the kernel fails to
  266. make progress in the specified amount of time.
  267. In these cases you attach the debugger in the same way: by running it
  268. and entering the <tt>target remote</tt> line.
  269. </p>
  270. <p>
  271. Importantly, you can also make System/161 stop and wait for the
  272. debugger by pressing ^G.
  273. This can be
  274. useful if your kernel is looping or deadlocked.
  275. </p>
  276. <p>
  277. Because typing that <tt>target remote</tt> command over and over is a
  278. major nuisance, we suggest setting up a macro for it.
  279. Edit the file <tt>~/cs161/root/.gdbinit</tt> and put the following in
  280. it:
  281. <pre>
  282. define db
  283. target remote unix:.sockets/gdb
  284. end
  285. </pre>
  286. On startup, GDB reads the file <tt>.gdbinit</tt> in the current
  287. directory and executes the GDB commands it finds in it; so now you can
  288. connect to System/161 by just typing
  289. <pre>
  290. (gdb) db
  291. </pre>
  292. It is also often useful to put common breakpoint settings in
  293. <tt>.gdbinit</tt>, such as on <tt>panic</tt> or <tt>badassert</tt>.
  294. (Note that as of OS/161 2.0, <tt>panic</tt> will itself automatically
  295. request a debugger connection.
  296. But you may find that putting an explicit breakpoint on it is still
  297. helpful.)
  298. </p>
  299. <p>
  300. Note that for reasons we don't yet understand (if we understood them,
  301. we have have fixed this),
  302. sometimes connecting the debugger while System/161 is running
  303. (rather than waiting for a debugger connection) doesn't work.
  304. Also sometimes, but not usually the same times, pressing ^G doesn't
  305. seem to work.
  306. (If you manage to get a repeatable test case for either of these
  307. conditions, contact dholland@eecs.harvard.edu.)
  308. </p>
  309. <p>
  310. Also note that if GDB is already connected, and you tell it to
  311. continue and the kernel doesn't stop again when you expected, getting
  312. it to stop so you can do more with GDB can be annoyingly difficult.
  313. Theoretically at this point pressing ^G in the System/161 window will
  314. cause it to stop executing and return control to GDB, but sometimes
  315. this doesn't work either.
  316. Hitting ^C in GDB will get you a GDB prompt again, but one where you
  317. can't do much of anything because the kernel is still running; GDB
  318. suggests you type <tt>interrupt</tt>, but this doesn't work.
  319. If all else fails, you can kill GDB, start another copy, and
  320. reconnect.
  321. Don't hit ^C in System/161 as that will kill it and lose your state.
  322. </p>
  323. <p>
  324. When you are done
  325. debugging, you can disconnect the debugger from System/161 (and thus
  326. the running kernel) in two ways:
  327. <pre>
  328. (gdb) detach
  329. </pre>
  330. will unhook the debugger and leave the kernel still running, whereas
  331. <pre>
  332. (gdb) kill
  333. </pre>
  334. will unceremoniously nuke System/161, much as if you'd gone to its
  335. window and typed ^C.
  336. Quitting out of gdb while connected is the same as using <tt>kill</tt>.
  337. </p>
  338. <p>
  339. Caution: Be sure that the kernel image that you run in System/161 and
  340. the one
  341. you run <tt>os161-gdb</tt> on are the same.
  342. If they aren't, it's like using a different edition of a textbook with
  343. different page numbers and not realizing it: things can get very
  344. bizarre and very confusing.
  345. </p>
  346. <h2>Executing under GDB control</h2>
  347. <p>
  348. Here are the most common and often-used commands for controlling the
  349. execution of the kernel in GDB:
  350. </p>
  351. <p>
  352. <b>s, step</b> - Step through the program.
  353. If you want to go through your program step by step after it has hit
  354. a breakpoint, use the "step" command.
  355. Typing
  356. <pre>
  357. (gdb) s
  358. </pre>
  359. will execute the next line of code.
  360. If the next line is a function call,
  361. the debugger will step into this function.
  362. </p>
  363. <p>
  364. <b>n, next</b> - Execute the next line.
  365. This command is similar to the "step" command, except that it does not
  366. step into a function, but executes the function, as if it were a simple
  367. statement. This is extraordinarily useful for stepping over functions
  368. such as kprintf or other functions that you know/believe to be correct.
  369. </p>
  370. <p>
  371. <b>adv, advance</b> - Advance to a specified line
  372. This is like "next" but continues to a specified line number.
  373. It is like setting a breakpoint on that line and using "continue".
  374. If you accidentally pick a line that you won't get to, it won't stop
  375. until the next breakpoint.
  376. </p>
  377. <p>
  378. <b>finish</b> - Continue until function return.
  379. This command advances until the end of the function.
  380. </p>
  381. <p><b>c, continue</b> - continue execution.
  382. When the kernel is stopped (at a breakpoint, or after connecting, or
  383. after executing manually for a while), type
  384. <pre>
  385. (gdb) c
  386. </pre>
  387. to just continue executing.
  388. Execution continues until something happens that stops it, such as
  389. hitting a breakpoint, or if you type ^G into System/161.
  390. (See above for some notes on ^G.)
  391. </p>
  392. <p>
  393. <b>b, break</b> - set a breakpoint.
  394. Use this command to specify that your program should stop execution at
  395. a certain line or function.
  396. Typing
  397. <pre>
  398. (gdb) b 18
  399. </pre>
  400. means that your program will stop every time it executes a statement
  401. on line 18.
  402. As with the "list" command, you can specify break on a function, e.g.:
  403. <pre>
  404. (gdb) b main
  405. </pre>
  406. will stop when execution encounters the main function.
  407. </p>
  408. <p>
  409. <b>info breakpoints</b> - List all breakpoints.
  410. </p>
  411. <p>
  412. <b>d, delete</b> - Delete a breakpoint by number.
  413. Use this command to delete a breakpoint.
  414. By typing
  415. <pre>
  416. (gdb) d 1
  417. </pre>
  418. you will delete the breakpoint numbered 1.
  419. GDB displays the number of
  420. a breakpoint when you set that breakpoint.
  421. Typing "d" without arguments
  422. deletes all breakpoints.
  423. </p>
  424. <p><b>clear</b> - Clear a breakpoint by name/address.
  425. If you don't remember the number of the breakpoint you want to delete,
  426. use the "clear" command.
  427. Just like the "breakpoint" command, it takes
  428. a line number or a function name as an argument.
  429. </p>
  430. <p>
  431. <b>disable, enable</b> - Disable/enable a breakpoint.
  432. Use these commands with a breakpoint number as an argument to disable
  433. or enable a breakpoint.
  434. When you disable a breakpoint, the breakpoint is still there, but execution
  435. will not stop at it until you enable the breakpoint.
  436. </p>
  437. <p>
  438. <b>cond</b> - Make a breakpoint conditional.
  439. This can be helpful if you need to set a breakpoint on a very common
  440. code path.
  441. Usage is like this:
  442. <pre>
  443. (gdb) cond 3 count > 1000
  444. </pre>
  445. which makes breakpoint 3 only happen when the expression
  446. <tt>count > 1000</tt> is true.
  447. CAUTION: in past years, we have found that use of conditional
  448. breakpoints seems to corrupt GDB's internal state such that it starts
  449. lying to you.
  450. This year (2016) we have a substantially newer GDB version that
  451. we hope no longer has this bug.
  452. Until we have further data, use conditional breakpoints cautiously,
  453. and let the course staff know what your experiences are.
  454. </p>
  455. <p>
  456. <b>command</b> - Execute a command on a breakpoint.
  457. You can specify that a
  458. certain command or a set of commands be executed at a breakpoint.
  459. For
  460. example, to specify that a certain string and a certain value are printed
  461. every time you stop at breakpoint 2, you could type:
  462. <pre>
  463. (gdb) command 2
  464. > printf "theString = %s\n", theString
  465. > print /x x
  466. > end
  467. </pre>
  468. </p>
  469. <h2>Inspecting stuff with GDB</h2>
  470. <p>
  471. The chief purpose of a debugger is to inspect the target program.
  472. Here are the most common/useful commands for doing that.
  473. </p>
  474. <p>
  475. <b>l, list</b> - List lines from source files.
  476. Use this command to display
  477. parts of your source file.
  478. For example, typing
  479. <pre>
  480. (gdb) l file.c:101
  481. </pre>
  482. will display line 101 in <tt>file.c</tt>.
  483. If you leave off the file name, GDB will use what it thinks is the
  484. current source file; this is usually the most recently referenced
  485. source file but occasionally isn't what one expects.
  486. You can also give a function name to list starting at the top of that
  487. function.
  488. And if you give no arguments GDB will list the next chunk of the file
  489. you last listed or if you just stopped execution, the point at which you
  490. stopped.
  491. </p>
  492. <p>
  493. Note: If you need to debug in <tt>start.S</tt> (or any other assembler
  494. file) GDB can't find the sources by default.
  495. This is because of a glitch in the toolchain: the paths to the
  496. assembler source files are relative, not absolute, so they only work
  497. by default in the kernel build directory.
  498. To work around it, you can tell GDB to start looking for the sources
  499. in the kernel build directory:
  500. <pre>
  501. (gdb) directory /home/jharvard/cs161/os161/kern/compile/ASSTN
  502. </pre>
  503. for whatever assignment <tt>N</tt> you're currently working on.
  504. Tip: you can put this in <tt>.gdbinit</tt>.
  505. </p>
  506. <p>
  507. <b>disassemble</b> - Disassemble some code.
  508. Use this command to print disassembly of the machine code for a
  509. function.
  510. Add <tt>/m</tt> to mix in the source listing as well.
  511. Hopefully you won't need to use this; but it can occasionally be
  512. useful.
  513. </p>
  514. <p>
  515. <b>x</b> - Examine memory.
  516. This is a general-purpose function for dumping memory; it has a
  517. variety of options and forms, all of which take a memory address
  518. (which can be the name of a variable or function) as an argument.
  519. The most useful are:
  520. <ul>
  521. <li><tt>x/x</tt> - dump memory as hex</li>
  522. <li><tt>x/4x</tt> - dump memory as hex four at a time</li>
  523. <li><tt>x/64x</tt> - dump memory as hex 64 at a time, which fits 256
  524. bytes nicely on a normal-size screen</li>
  525. <li><tt>x/s</tt> - dump memory as null-terminated strings</li>
  526. <li><tt>x/c</tt> - dump memory as single characters</li>
  527. <li><tt>x/i</tt> - dump memory as disassembly</li>
  528. </ul>
  529. but there are many more variations; see the GDB help on the command
  530. for the full details.
  531. </p>
  532. <p>
  533. <b>bt, where</b> - Display backtrace.
  534. To find out where you are in the execution of your program and how you
  535. got there, use one of these commands.
  536. This will show the backtrace of
  537. the execution, including the function names and arguments.
  538. </p>
  539. <p>
  540. <b>up</b>, <b>down</b> - Navigate the backtrace.
  541. You can move up the stack to parent stack frames to inspect things
  542. there.
  543. With both the "up" and "down" commands you can give an optional
  544. numeric argument to move up or down that many frames instead of just
  545. one.
  546. </p>
  547. <p>
  548. <b>frame</b> - Select a stack frame in the backtrace.
  549. With a numeric argument, this moves up (or down, as needed) the stack
  550. trace to the frame with the specified number.
  551. (Frame 0 is the innermost, where execution is actually occurring.)
  552. Without an argument it reselects the last frame you navigated to,
  553. which both prints it out and has the often-useful side effect of
  554. resetting the current listing position to there.
  555. </p>
  556. <p>
  557. <b>p, print</b> - Print an expression.
  558. Use for example
  559. <pre>
  560. (gdb) p name
  561. </pre>
  562. to print the value of <tt>name</tt>.
  563. You can use arbitrarily complex C expressions; it is often useful for
  564. example to include typecasts.
  565. Normally the value will be printed according to its type; you can
  566. insert insert <tt>/x</tt>, <tt>/s</tt>, <tt>/d</tt>, <tt>/u</tt>
  567. to force printing as hex, as a string, or as a signed or unsigned
  568. integer respectively.
  569. </p>
  570. <p>
  571. When you print something, you'll get a result line like this:
  572. <pre>
  573. $1 = 0x80063d80 ""
  574. </pre>
  575. The <tt>$1</tt> is a <em>convenience variable</em>, a scratch variable
  576. created and remembered by GDB for your subsequent use.
  577. You can use <tt>$1</tt> as a value in subsequent expressions and it
  578. will retrieve the value that was printed; this saves having to retype
  579. it or cut and paste it.
  580. (Note though that it saves the <em>value</em>, not the
  581. <em>expression</em>; if you execute for a while and then use
  582. <tt>$1</tt> again it will have the same value regardless of what's
  583. happened in the meantime to the values in the expression that
  584. generated it.
  585. </p>
  586. <p>
  587. <b>printf</b> - Formatted print.
  588. The "printf" command allows you to specify the formatting of the output,
  589. just like you do with a C library printf() function.
  590. For example, you
  591. can type:
  592. <pre>
  593. (gdb) printf "X = %d, Y = %d\n",X,Y
  594. </pre>
  595. </p>
  596. <p>
  597. <b>info registers</b> - Show the processor registers.
  598. This prints all the processor registers.
  599. You can also use values from registers in "print" expressions:
  600. a <tt>$</tt> followed by the symbolic name of the register fetches
  601. that register's value; e.g. the return value of a function is normally
  602. left in <tt>$v0</tt>.
  603. </p>
  604. <p>
  605. <b>display</b> - Display an expression.
  606. Compute and display the value of an expression every time the program
  607. stops.
  608. For example:
  609. <pre>
  610. (gdb) display abc
  611. </pre>
  612. If there is no <tt>abc</tt> in scope when the program stops, the
  613. display won't appear.
  614. This command is otherwise the same as "print", including the support
  615. for type modifiers.
  616. </p>
  617. <p>
  618. <b>undisp</b> - Remove a "display".
  619. The argument is the number of the "display" expression to remove.
  620. With no argument, all "display" expressions are removed.
  621. Also, <tt>delete display N</tt> is the same as <tt>undisp N</tt>.
  622. </p>
  623. <p>
  624. <b>set variable</b> - Assign a value to a variable.
  625. Sometimes it is useful to change the value of a variable while the
  626. program is running.
  627. For example if you have a variable "x", typing
  628. <pre>
  629. (gdb) set variable x = 15
  630. </pre>
  631. will change the value of "x" to 15.
  632. </p>
  633. <h2>Other GDB commands to note</h2>
  634. <p>
  635. <b>define</b> - Define a new command.
  636. Use this if you want to define a new command.
  637. This is similar to assigning
  638. a macro and can save you typing.
  639. Just as with a macro, you can put
  640. together several commands.
  641. An example of defining a short form of the <tt>target remote</tt>
  642. command for attaching to System/161 was provided above.
  643. Some other more elaborate examples can be found in the
  644. <tt>gdbscripts</tt> directory in the OS/161 source tree.
  645. </p>
  646. <p>
  647. <b>help</b> - Get help (on gdb, not your CS161 assignments!)
  648. To find more about a particular command just type:
  649. <pre>
  650. (gdb) help &lt;command name&gt;
  651. </pre>
  652. Note that several of the commands listed above (e.g. "info
  653. breakpoints") are specific instances of more general GDB commands,
  654. like "info" and "set".
  655. </p>
  656. <h2>GDB threads support</h2>
  657. <p>
  658. Nowadays GDB understands the concept of multiple threads; you can
  659. switch from one thread to another while debugging in order to inspect all the
  660. available execution contexts.
  661. The good news is: this now works with System/161.
  662. The not so good news is: GDB threads map to CPUs, not to OS/161 kernel
  663. threads.
  664. This means that while you can inspect all the CPUs that are executing,
  665. you still can't easily inspect sleeping threads.
  666. Unfortunately, GDB's thread model combined with the fact that the
  667. debugger talks transparently to System/161 makes this mapping more or
  668. less the only choice.
  669. </p>
  670. <p>
  671. When stopped in GDB, list the threads like this:
  672. <pre>
  673. (gdb) info threads
  674. </pre>
  675. This will give a display something like this:
  676. <pre>
  677. Id Target Id Frame
  678. 4 Thread 13 (CPU 3) wait () at ../../arch/mips/thread/cpu.c:279
  679. 3 Thread 12 (CPU 2) wait () at ../../arch/mips/thread/cpu.c:279
  680. 2 Thread 11 (CPU 1) wait () at ../../arch/mips/thread/cpu.c:279
  681. * 1 Thread 10 (CPU 0) runprogram (progname=0x800f3f40 "/bin/sh")
  682. at ../../syscall/runprogram.c:492
  683. </pre>
  684. The leftmost column is the thread number that you need to use while
  685. talking to GDB, and the CPU number listed is the System/161 CPU
  686. number.
  687. Unfortunately the GDB thread numbers are offset by 1 at the start and
  688. may diverge further over time at GDB's whim.
  689. (The "Thread 13" number is the number used in the communications
  690. channel between GDB and System/161; you don't need to care about it.
  691. It's offset from the CPU number by 10 because if you use 0 GDB dumps
  692. core.)
  693. </p>
  694. <p>
  695. To switch to another CPU, use the "thread" command with the number
  696. from the leftmost "Id" column:
  697. <pre>
  698. (gdb) thread 1
  699. </pre>
  700. </p>
  701. <h2>The GDB "text user interface"</h2>
  702. <p>
  703. GDB has a curses-based "text user interface" (TUI) that gives you a
  704. slightly less 1980 experience when debugging.
  705. This gives you a split-screen window with the current source file in
  706. the top part and the GDB prompt in the bottom part.
  707. Type <tt>^X a</tt> (that's control-X followed by the letter a)
  708. after starting GDB to
  709. enable it, and type that again to make it go away.
  710. </p>
  711. <p>
  712. In TUI mode most GDB commands work as before; there are just a
  713. couple of keystrokes you need to know about:
  714. <ul>
  715. <li>^L - redraw screen (like in many programs)</li>
  716. <li>^X o - switch to other panel in the GDB window (like in Emacs)</li>
  717. </ul>
  718. By default the current panel is the source listing, but typing goes
  719. into the GDB prompt panel anyway; however, to use the arrow keys for
  720. input editing you need to switch to the GDB prompt panel.
  721. </p>
  722. <p>
  723. There is also a graphical front end to GDB called DDD; it should be
  724. possible to use this with the OS/161 version of GDB.
  725. No current information on this is available right now, unfortunately.
  726. </p>
  727. <h2>Debugging tips</h2>
  728. <p>
  729. Tip #1: Check your beliefs about the program
  730. </p>
  731. <p>
  732. So how do you actually approach debugging? When you have a bug in a
  733. program, it means that you have a particular belief about how your program
  734. should behave, and somewhere in the program this belief is violated.
  735. For
  736. example, you may believe that a certain variable should always be 0 when
  737. you start a "for" loop, or a particular pointer can never be NULL in
  738. a certain "if statement".
  739. To check such beliefs, set a breakpoint in
  740. the debugger at a line where you can check the validity of your belief.
  741. And when your program hits the breakpoint, ask the debugger to display
  742. the value of the variable in question.
  743. </p>
  744. <p>
  745. Tip #2: Narrow down your search
  746. </p>
  747. <p>
  748. If you have a situation where a variable does not have the value you
  749. expect, and you want to find a place where it is modified, instead of
  750. walking through the entire program line by line, you can check the value
  751. of the variable at several points in the program and narrow down the
  752. location of the misbehaving code.
  753. </p>
  754. <p>
  755. Tip #3: Walk through your code
  756. </p>
  757. <p>
  758. Steve Maguire (the author of Writing Solid Code) recommends using
  759. the debugger to step through every new line of code you write, at
  760. least once, in order to understand exactly what your code is doing.
  761. It helps you visually verify that your program is behaving more or less
  762. as intended.
  763. With judicious use, the step, next and finish commands
  764. can help you trace through complex code quickly and make it possible to
  765. examine key data structures as they are built.
  766. </p>
  767. <p>
  768. Tip #4: Use good tools
  769. </p>
  770. <p>
  771. Using GDB with a visual front-end can be very helpful.
  772. For example,
  773. using GDB inside the emacs editor puts you in a split-window mode,
  774. where in one of the windows you run your GDB session, and in the other
  775. window the GDB moves an arrow through the lines of your source file as
  776. they are executed.
  777. To use GDB through emacs do the following:
  778. <ol>
  779. <li> Start emacs.
  780. <li> Type the "meta" key followed by an "x".
  781. <li> At the prompt type "gdb". Emacs will display the message:
  782. <pre>
  783. Run gdb (like this): gdb
  784. </pre>
  785. <li> Delete the word "gdb", and type:
  786. <pre>
  787. os161-gdb kernel
  788. </pre>
  789. So in the end you should have:
  790. <pre>
  791. Run gdb (like this): os161-gdb kernel
  792. </pre>
  793. displayed in the control window.
  794. </ol>
  795. </p>
  796. <p>
  797. At this point you can continue using GDB as explained in section 2.
  798. </p>
  799. <p>
  800. Tip #5: Beware of printfs!
  801. </p>
  802. <p>
  803. A lot of programmers like to find mistakes in their programs by inserting
  804. "printf" statements that display the values of the variables.
  805. If you
  806. decide to resort to this technique, you have to keep in mind two things:
  807. First, because adding printfs requires a recompile, printf debugging
  808. may take longer overall than using a debugger.
  809. </p>
  810. <p>
  811. More subtly, if you are debugging a multi-threaded program, such as
  812. a kernel, the order in which the instructions are executed depends on
  813. how your threads are scheduled, and some bugs may or may not manifest
  814. themselves under a particular execution scenario.
  815. Because printf outputs
  816. to the console, and the console in System/161 is a serial device that
  817. isn't extraordinarily fast, an extra call to printf may alter the timing
  818. and scheduling considerably.
  819. This can make bugs hide or appear to come
  820. and go, which makes your debugging job much more difficult.
  821. </p>
  822. <p>
  823. To help address this problem, System/161 provides a simple debug output
  824. facility as part of its trace control device.
  825. One of the trace control
  826. device's registers, when written to, prints a notice in the System/161
  827. output including the value that was written.
  828. In OS/161, provided your
  829. System/161 has been configured to include the trace control device, you
  830. can access this feature by calling trace_debug(), which is defined in
  831. dev/lamebus/ltrace.h.
  832. While this is less disruptive than calling printf,
  833. it is still not instant and can still alter the timing of execution.
  834. By
  835. contrast, the System/161 debugger interface is completely invisible; as
  836. far as your kernel is concerned, time is stopped while you are working
  837. in the debugger.
  838. </p>
  839. <p>
  840. Tip #6: Debugging deadlocks
  841. </p>
  842. <p>
  843. A deadlock occurs when two or more threads are all waiting for the
  844. others to proceed.
  845. (We'll talk more about this in lecture.)
  846. In a simple deadlock there'll be one thread T1 that already holds a lock L1,
  847. and another thread T2 that already holds a lock L2.
  848. T1 is waiting to get L2, so it can't proceed until T2 runs, and T2 is
  849. waiting to get L1, so it can't proceed until T1 runs.
  850. The goal of debugging a deadlock is to find out the identities of T1
  851. and T2, and of L1 and L2, and then figure out what combination of
  852. circumstances and code paths allowed them to get into this state.
  853. </p>
  854. <p>
  855. To do this, you generally want to begin by finding one of the threads,
  856. since there are usually a lot more locks in the system than threads.
  857. Each thread that's blocked reports what it's blocked on in the field
  858. <tt>t_wchan_name</tt>.
  859. The basic procedure is to locate a thread (the first ones to start
  860. with are the ones currently on the CPUs; after that look in your
  861. process table that you wrote for assignment 2), check what it's
  862. waiting for, then find who's holding that.
  863. If it's waiting for a lock, hopefully the locks you wrote record who's
  864. holding them.
  865. That gives you another thread to examine; repeat until you find a
  866. loop.
  867. If it's waiting on a CV, it gets a bit more interesting because you
  868. need to figure out who was supposed to signal that CV and didn't... or
  869. if they did, why the waiting thread didn't receive the signal.
  870. For deadlocks involving spinlocks, remember that in OS/161 spinlocks
  871. are held by CPUs, not threads.
  872. </p>
  873. <p>
  874. Once you have the participants identified, the stack traces from those
  875. threads will usually (but, alas, not always) give you enough
  876. information to work out what happened.
  877. </p>
  878. <p>
  879. It is often helpful to set up a global table of all locks and/or CVs
  880. (OS/161 already comes with a global table of all wchans) to make this
  881. process easier.
  882. It is also possible, and potentially worthwhile, to hack more state
  883. information into the thread structure.
  884. One can even write an automatic deadlock detector, although this
  885. probably has a bad cost/benefit ratio and won't help you much when CVs
  886. are involved.
  887. </p>
  888. <p>
  889. Tip #7: Debugging assembly
  890. </p>
  891. <p>
  892. When GDB stops in an assembly source file (a .S file) various special
  893. considerations apply.
  894. GDB is meant to be a source-level debugger for high
  895. level languages and isn't very good at debugging assembly.
  896. So various
  897. tricks are required to get adequate results.
  898. </p>
  899. <p>
  900. The OS/161 toolchain tells the assembler to emit line number
  901. information for assembly files, but as noted above (under "list") you
  902. need to tell GDB how to find them.
  903. Do that.
  904. Then you can at least see the file you're working on.
  905. </p>
  906. <p>
  907. It is also sometimes helpful to disassemble the kernel; type
  908. <pre>
  909. % os161-objdump -d kernel | less
  910. </pre>
  911. in another window and page or search through it as needed.
  912. </p>
  913. <p>
  914. To single step through assembler, use the nexti and stepi commands,
  915. which are like next and step but move by one instruction at a time.
  916. </p>
  917. <p>
  918. The command x /i (examine as instructions) is useful for disassembling
  919. regions from inside GDB.
  920. </p>
  921. <p>
  922. Print register values with <tt>$</tt> and the symbolic register names
  923. as described above (<tt>$v0</tt>, <tt>$a0</tt>, etc.) to see the
  924. values that are being
  925. handled.
  926. </p>
  927. <p>
  928. Tip #8: trace161
  929. </p>
  930. <p>
  931. The trace161 program is the same as sys161 but includes additional
  932. support for various kinds of tracing and debugging operations.
  933. You can
  934. have System/161 report disk I/O, interrupts, exceptions, or whatever.
  935. See
  936. the System/161 documentation for more information.
  937. </p>
  938. <p>
  939. One of the perhaps more interesting trace options is to have System/161
  940. report every machine instruction that is executed, either at user level,
  941. at kernel level, or both.
  942. Because this setting generates vast volumes
  943. of output, it's generally not a good idea to turn it on from the command
  944. line.
  945. (It is sometimes useful, however, in the early stages of debugging
  946. assignment 2 or 3, to log all user-mode instructions.)
  947. However, the
  948. trace options can be turned on and off under software control using
  949. the System/161 trace control device.
  950. It can be extremely useful to turn
  951. instruction logging on for short intervals in places you suspect something
  952. strange is happening.
  953. See dev/lamebus/ltrace.h for further information.
  954. </p>
  955. <p>
  956. Tip #9: casting void pointers
  957. </p>
  958. <p>
  959. If you have a void * in GDB and you know what type it actually is, you
  960. can cast it when printing, using the usual C expression syntax.
  961. This is very helpful when dealing with arrays, for example.
  962. </p>
  963. <p>
  964. Tip #10: tracing through exceptions
  965. </p>
  966. <p>
  967. When you get a stack backtrace and it reaches an exception frame, GDB can
  968. sometimes now trace through the exception frame, but it doesn't always
  969. work very well.
  970. Sometimes it only gets one function past the exception,
  971. and sometimes it skips one function.
  972. (This is a result of properties of
  973. the MIPS architecture and the way GDB is implemented and doesn't appear
  974. readily fixable.)
  975. Always check the tf_epc field of the trap frame to
  976. see exactly where the exception happened, and if in doubt, cross-check
  977. it against a disassembly or have GDB disassemble the address.
  978. </p>
  979. <h2>Where to go for help</h2>
  980. <p>
  981. For help on GDB commands, type "help" from inside GDB.
  982. You can find the
  983. <a href="http://www.gnu.org/software/gdb/gdb.html">GDB manual here</a>
  984. and of course your friendly TFs are always there to help!
  985. </p>
  986. </body>
  987. </html>