123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634 |
- @make(Article)
- @comment[
- 9-Dec-82 20:46:50-MST,16664;000000000001
- Mail-from: ARPANET site RAND-RELAY rcvd at 9-Dec-82 2044-MST
- Date: 9 Dec 1982 0544-PST
- From: GRISS.HP-HULK at Rand-Relay
- Subject: Draft of more BUILD
- To: jw-peterson at Utah-20
- Via: HP-Labs; 9 Dec 82 19:36-PDT
- Here is a portion of manual about the next steps. Not complete,
- incorp@orates some of what youve seen:
- ]
- [For the moment, this note will use 68000 building as example, using
- DEC-20 as HOST]
- @section(Building the Cross Compiler)
- Connect to P68c: .
- Make sure that you have the following .b files on p68c:, or rebuild
- as below:
- a. m68k-cmac.b
- b. m68k-comp.b
- c. m68k-asm.b
- @subsection(How to make the .B files using the .MIC files)
- To rebuild a missing .B file, run the SYSBUILD .MIC file
- on the appropriate module:
-
- @@MIC PU:sysbuild M68k-xxx
- where "xxx" represents CMAC, COMP or ASM, as appropriate.
- @subsection(How to make the .EXE file)
- Now build the cross compiler onto the scratch directory, S: by
- running the .CTL file (using DO or SUBMIT):
-
- @@do p68c:new-m68k-cross.ctl
- [In the future, this should actually be changed to "do
- new-apollo-cross" to avoid confusion between the various 68000 based
- machines].
- @section(Running the Cross Compiler)
- Now connect to p68:
- @subsection(Independent Compilation and the .SYM file)
- To build a fresh BARE-PSL or a fresh FULL-PSL you will need a fresh
- symbol file. The current symbol file has the name of "m68k.sym"
- [which should be changed to "apollo.sym" or something related in the future]
- First generate a fresh m68k.sym file:
- @@MIC fresh-kernel.ctl
- This will keep your last m68k.sym file as p68:previous-m68k.sym. The
- fresh m68k.sym file will be on S:. Make sure it is there.
- @subsection(Generating the Module .CTL Files)
- Currently fourteen modules are required to build the first phase of
- either the
- BARE-PSL build or the FULL-PSL. You will need xxx.CTL files on kapollo:
- for each of these. The kernel module names (xxx) are currently:
- <<< how's about compiler modules? have same faclity make .ctl files for
- those?? >>>
- TYPES RANDM ALLOC ARITH DEBG ERROR EVAL EXTRA FASL
- IO MACRO PROP
- SYMBL SYSIO TLOOP HEAP
- and
- MAIN
- [Note, order is different from older P68: version <<<how??>>>]
- [Note, change to generate also for BIG-KERNEL?]
- <<< there needs to be some clear consensous on the terminolgy, i.e., what's the
- differances between big/bare/full-comp/kernel/psl??? >>>
- Take a look to see if they are there, if they are not
- you will have to re-generate them. The easiest way will be to do this
- is via the "kernel-gen" program:
- @begin(verbatim)
- @@PSL:PSL
- *(dskin "apollo-kernel-gen.sl")
- *(quit)
- @end(verbatim)
- This will create the xxx.CTL files you need on kapollo:.
-
- @subsection(Building the Modules)
- <<<again, terminology. need some clear definitions as to what all
- encompases (in terms of functionality, not "contains xx, yy & zz") >>>
- Now connect to kapollo:
- Now you must execute all the CTL files for the first 14 modules. Do this with
- the following command:
- @@MIC kapollo:All-kernel.ctl
- This command will SUBMIT all these CTL files to batch.
- [Alternatively, single modules my be run by submitting
- @@SUBMIT xxxx.CTL
- for module xxx] <<< any order or presatance to be followed? hows about .sym
- file??>>>
- Each batch job processed will create an xxx.log file on kapollo: which
- you can look at to evaluate errors. Initially before running a fresh
- build you might want to delete all these log files just for the sake
- of space.
- @subsection(Processing the MAIN file)
- <<<re: "is built last" where (timewise) does the compiler fit in?>>>
- Note that the MAIN module is built last, and that it takes the
- contents of the .SYM file and builds the run-time symbol table
- initialization.
- @@submit MAIN.CTL
- [Why is this not in ALL-KERNEL.CTL?]<<<because all-kernel refers to building
- the Individual pieces. Main crunches specificly on the main-start file and
- builds a resulting dmain. it is separate from all-kernel (specifcly, if i
- remember) simply so it Can be run last>>>.
- @subsection(Linking the files and executing)
- <<<huh?>>>
- @section(Details on the Test series)
- [Absorb details from TEST GUIDE here]
- <<< NO! we're talking about building *re*building sources that are assumed to
- be complete (i.e., a new version), not developing a port to a new machine.
- the port process, including the use of the small tests, deserves to be in
- a separate document; as it works quite differntly from building the whole
- thing. >>>
- @subsection(Command Files, and Kernel Generator?)
- [Describe kernel generator earlier?] <<< yes, please. and while you're at
- it, a functional description of "a kernel", and what it must contain, would
- help.>>>
- @subsection(Basic Test Strategy)
- Each test will use some modules tested in previous test,
- and add others, mostly extracted from the full PSL sources.
- Occassionally some stub-files have to be added, to be replaced
- by more complete sets extracetd frm sources later. Early tests
- simply try to print informative messages about what is happening,
- and whether each test succeeded or not. As more of a complete
- LISP is built up, the tests will require a variety of manual
- inputs. Finally a complete MINI-PSL will result.
- <<< again, i'd like to see the porting manual separate from the system-rebuild
- description; not doing so risks confusion, and perhaps a 'missing piece
- syndrome'. the idea is pick up vol. one "how to design and test psl cmacros"
- once you think your cmacros work, you pick up vol. two "how to build a
- complete running psl." theoreticly, the only thing in common between the
- two should be one(?) i/o module and the key compiler files xxx-cmac, xxx-asm,
- etc. (they guys who live in .../comp) >>>
- @subsection(Test1)
- @subsection(Testn)
- @subsection(Testing Mini-EVAL)
- @subsection(Testing Character and File I/O)
- @subsection(Switch Over to INIT files)
- <<< what switch? where? magic? >>>
- At this point, can flip a switch in the build process, and
- have INITCODE be smaller, and instead have .INIT files produced,
- which will be read in by LAPIN or DSKIN.
- @subsection(Testing Binary I/O)
- [Write a small BINDUMP routine]
- <<<again, the vol.1 "how to test"/vol 2."how to build" concept. perhaps
- set up a testn+1 to test bin i/o?? >>>
- @section(Building the BARE-PSL kernel)
- At this point, enough basic tests have been done, and now the standard
- BARE-PSL should be built. This requires a few more files,
- <<<this is where things can get murky between "test phase" and "build phase".>>
- and a more
- stable BUILD sequence. This will result in a complete 3.1 version of
- BARE-PSL.
- <<<what about comp/faslout? build it on the resident bare-psl via the
- interpreter? maybe go whole-hog first time? we thought we could get away
- bare-psl on the apollo mainly because we thought the 3.0 could handle
- generating new binaries. it couldn't, so we had start from square 0. and
- you can't (at least if i interpeted all of chip & steve's swearing & cursing
- right) build the comp stuff interpetivly because you start tripping
- over syslisp. is that now fixed? if so, how? needs
- the concept as presented here needs details, and looks like it may not
- be fully correct...take a hard look.>>>
- @subsection(Use and Customization of Kernel Generator)
- [Should kernel-gen be used with test series?]
- <<<no, see above dissertation on vol1/vol2>>>
- @subsection(Common Files, Machine Specific Files and Dummy Files)
- @subsection(Init Files)
- <<<short section. I could use the info, what -are- they used for?
- when do you need to replace them?>>>
- @subsection(Testing BARE-PSL)
- @section(Bootstrapping the LAP, FASL and COMPILER)
- Currently, we bootstrap complete system by adding additional modules
- to BARE-PSL to make BIG-PSL.<<<terminology again>>>
- These are LAP, FASLOUT and COMPILER
- modules, and also RLISP parser. BIG-PSL <<<don't you mean bare?>>>
- is used as a bootstrap step to
- the production of COMPILER.B, FASLOUT.B, LAP.B etc., since once these
- are built, they can be loaded into the BARE-PSL when needed.
- Having core-save working by this time is important, since
- the kernel is quite large, and loading RLISP and COMPILER and INIT
- files takes quite a while. <<<though somewhat of a moot point on the apollo,
- since copying the entire image also takes plenty of time>>>.
- [In future, should convert critical files to .SL, avoid RLISP in
- kernel at ALL] <<<or how's about the host generating a .sl rlisp
- automaticlly? I would Much rather read .red then .sl >>>
- [In future, will do alternative model, with just LAP to start, test
- with LAP files from cross-compiled files. Then test FASLOUT and
- FASLIN. Should be able to load many things as .LAP files. Then
- finally load compiler. It should work without much problem since its
- essentially all common code, and mostly tested even for this target in
- CROSS mode.] <<<yeah. reminds me, this doc doesn't say much about lap.
- generation of the lap system is quite arcane, no?>>>
- @subsection(Building the FULL-PSL)
- Essentially same procedure as BARE-PSL, just have 2 more modules,
- RLISP and COMP, and rebuild MAIN. <<<but if you're going the cross compile
- route, watch out for booby traps (i.e., fasl in bare-psl stepping on fasl
- in comp>>>
- @subsection(Extra Files)
- For the RLISP module, need PU:RLISP.BUILD which accesses
- PU:RLISP-PARSER.RED and PU:RLISP-SUPPORT.RED.
- [We should change sources so that dont need RLISP for
- for BIG-BUILD].
- For the COMP module, we need to access a large number of
- files right now: <<<huh? this is mislocated>>>
- @subsection(Building both BARE-PSL and FULL-PSL)
- Its worth building both BARE-PSL and FULL-PSL at the same time during this
- phase. Build up to the MAIN module of BARE-PSL. Then copy the .SYM
- file for use in incremental rebuilding of BARE-PSL modules and
- BARE-MAIN. Then continue to build the RLISP, COMP and FULL-MAIN
- modules. These 2 different .SYM files are then used for rebuilding
- modules in the BARE-PSL series or the FULL-PSL series, as appropriate.
- Most of the time, errors will be only in the COMP module, but occasionally
- errors will be found that require a full build of the BARE-PSL and FULL-PSL,
- or incremental rebuild of some earlier modules. <<<hmmm, what about .sym
- file? and cleaning it out and restoring it? and how do the .init files
- fit into this process. i don't like the idea of several lisps lying around
- (e.g., bare, big, full, etc).
- would be MUCH simpler just to deal with one resulting system, rather than try
- and keep track of several. particularly if they start getting into fights
- and stepping on each other. cost in dealing with one larger system may be made
- up in avoiding screwups caused by multpile ones. think about this!>>>
- To build a FULL-PSL you must submit two additional .CTL files to be
- cross compiled, they are COMP.CTL and MAIN.CTL. To build just BARE-PSL
- you submit only MAIN.CTL. Both of these CTL files should be on
- kapollo:, if not you will have to create them by hand.
- Here is COMP.CTL:
- @begin(verbatim)
- @@define DSK: DSK:, kapollo:, PI: <<<search lists are too much a form of
- magic. would prefer that it be
- dictated as to which dir the .ctl is
- run from, and logicals (or on unix,
- relative paths) be used to specify
- where things belong. besides, they
- Only work this way on the 20.>>>
- @@S:m68k-CROSS.EXE
- *ASMOut "comp";
- *in "comp.build";
- *ASMEnd;
- *quit;
- The COMP.BUILD file should look like this:
- macro procedure !* u;nil;
- on eolinstringok;
- put('bitsperword,'wconst,32);
- compiletime flag('(taggedlabel inump !*jumpeq !*jumpnoteq
- !*jumpwgreaterp !*jumpwlessp !*jumpwgeq
- !*link !*linke
- onep
- !*jumpwleq), 'lose);
- in "pc:anyreg-cmacro.sl"$
- in "pc:common-cmacros.sl"$
- in "pc:common-predicates.sl"$
- in "pc:pass-1-lap.sl"$
- in "pc:compiler.red"$
- in "pc:comp-decls.red"$
- in "pc:tags.red"$
- compiletime remflag('(taggedlabel inump !*jumpeq !*jumpnoteq
- !*jumpwgreaterp !*jumpwlessp !*jumpwgeq
- !*link !*linke
- !*jumpwleq), 'lose);
- compiletime flag('(tagnumber), 'lose);
- in "kapollo:m68k-cmac.sl"$
- in "kapollo:m68k-comp.red"$
- in "kapollo:m68k-lap.red"$
- in "p68:nsystem-faslout.red"$ <<<are these duplicated in the bare-kernel?>>>
- in "pc:faslout.red"$ <<<again, problems with multilpe version, maybe not
- a good idea>>>
- The MAIN.CTL file will look like this:
- define DSK: DSK:, PHP:, PI:
- S:HP-CROSS.EXE
- ASMOut "main";
- in "main.build";
- ASMEnd;
- quit;
- @end(verbatim)
- So send one or both of these files to batch like this
- "submit comp.ctl"
- "submit main.ctl"
- Each ctl file sent to batch will produce three files on the scratch
- directory, an xxx.ASM, an Dxxx.ASM, and a xxx.INIT file.
- Some of the init files are of length zero, this is ok.
- @subsection(Append INIT files)
- Connect to the scratch directory, S:.
- The init files can all be appended together to cut down shipping and the
- time it takes to startup the APOLLO PSL.
- Append all the init files together to create an all.init.
- If you also are building the BIG-PSL then you will have to append
- COMP.INIT to all.init by hand or ship it to the apollo seperately and
- edit the file on the Apollo to include the comp.init.
- @@DO P68:all-init.ctl
- @subsection(Removing Tabs)
- [I believe 3.1 CROSS compiler fixed to only put in 1 space (or 2 for CRAY),
- so tabs dont need to be stripped. EXPAND is unsafe program]
- The Apollo Assembler does not like tabs so the .ASM files will need to
- have the tabs expanded into spaces. One way to do this is to do the
- following.
-
- @@DO p68:allexpand.ctl <<<unix has much better facilities for doing this>>>
- If you are building a BIG-PSL then you will have to expand the two comp
- by hand by doing:
- @@unix:expand <comp.asm >comp.asm
- @@unix:expand <dcomp.asm >dcomp.asm
- I suggest you copy everything to rs: to keep it around. Thats all
- the .asm's, the .inits, and the m68k.sym.
- [Why not change the .CTL files to insert RS: instead of S:]
- <<<perhaps because disk space is guarenteed on scratch, i.e., an extra
- set of versions won't kill you. would be nice tohave them back the next
- day though....>>>
- @subsection(Ship via the VAX)
- You are now ready to ship the code to the Apollo. Login on the VAX
- and run
- regexp.csh,
- a copy is on lowder's directory. This will move all the files off
- scratch except for the two comp files. So do:
- [Add BIGregexp.csh] <<<what on earth does regexp stand for?>>>
- <<<important: you should also give the following vax commands to avoid
- getting screwed over by mail, system, and autologout msgs:
- biff n #shut off mail notifyier
- mesg n # sys msgs
- set autlogout=2000 #so it won't die while waiting for asm
- >>>
- get20 scratch comp.asm dcomp.asm
- @subsection(Fetch from Apollo)
- Get logged in on the Apollo and conect to the VAX by running ST.
- >From the Apollo shell type:
- "apollo.csh"
- This will ship and assemble everything from the VAX except files related
- to comp. If you are using them you will have to type this to the apollo:
-
- [Add BIGAPOLLO.csh]
- "vfv1 comp.asm
- asmnl comp
- vfv1 dcomp.asm
- asmnl dcomp"
- @subsection(Bind the Modules)
- Now link with shell script:
- PSLBIND.SH PSL
- [Here again you CURRENTLY have to edit pslbind.sh to add the names of
- COMP.BIN and DCOMP.BIN if you are going to build a BIG-PSL. Suggest
- doing this once, create a BIGBIND.SH] <<<again, look at the special
- casy-ness of having big vs. bare [vs. full], etc. worth avoiding?
- time savings in the long run?>>>
- @subsection(Notes)
- There are a number of ways to vary this entire prcocess to customize
- it to your needs. If you started by building a BARE-PSL you can go
- back and build just the comp module by copying the m68k.sym from rs:
- onto the scratch directory and submitting the comp.ctl and the the
- main.ctl as previously described. Also you can choose to link or not
- the comp module in the apollo.
- <<<important: you need to spell out booby traps you can run into while
- doing this>>>
- @subsection(Testing LAP)
- Once most of LAP has been run on the host machine (interpretively or
- compiled), the next step is to run it as a "resident" PSL assembler on
- the target machine to ensure that it correctly assembles small
- procedures written in TLM ("target" LAP) form. Then procedures are
- input in ALM (cmacro form). Usually this next step will work quite
- well, since the CMACRO's will have been well tested while building the
- TEST-SERIES and BARE-PSL.
- Note that until RESIDENT mode of assembly seems stable (basically
- checking assembler and cmacro tables), there is no point in trying
- to do much with faslout.
- Here are some simple procedures to try; others can be generated
- by looking at the output of the cross-compiler:
- <<<comments! what are these guys trying to do? what should i look for
- to see that they work right? >>>
- @begin(verbatim)
- (LAP '((!*ENTRY FOO EXPR 1) % can we define ANY procedure
- (!*ALLOC 0)
- (!*EXIT 0))) % or (RTS) on 68000
- % when called, should return argument
- (LAP '((!*ENTRY FOO EXPR 0)
- (!*ALLOC 0)
- (!*MOVE (QUOTE 1) (REG 1))
- (!*EXIT 0)))
- (LAP '((!*ENTRY FOO EXPR 1) % adds 2 to argument, prints and returns
- (!*ALLOC 0)
- (!*MOVE (QUOTE 2) (REG 2))
- (!*LINK PLUS2 EXPR 2)
- (!*LINK PRINT EXPR 1)
- (!*EXIT 0)))
- @end(verbatim)
- Common problems encountered at this phase are:
- @begin(description)
- LAP Table Errors@\Most implementations of lap have procedures
- for common formats, and tables of numbers for the opcodes.
- Often the numbers are mistyped, or the instructions misclassified
- or missing.
- Trace@\If it blows up with illegal addressing, try tracing certain passes
- to see which is at fault; then as a quick patch, redefine these
- passes to be NO-OPS:
- @begin(verbatim)
- (de OptimizeBranches (U) U)
- or
- (de PASS1LAP (u) U)
- etc.
- @end(verbatim)
- @end(description)
- <<<what does alm mean?>>>
- [Prepare file of sample procedures, and corresponding ALM form
- to test important things. E.g., HALFWORD tables for LAMBIND, etc.]
- <<< why did chip & steve use interpretiv put/gethalfword functions?
- tricks worth knowing about???>>>
- [In future, hope to be able to run LAP interpretively on BARE-PSL,
- rather than having to build into kernel.]
- @subsection(Testing FASLOUT and FASLIN)
- Now that resident LAP seems to work, try some simple FASLOUT and
- FASLIN. Binary I/O should have been tested, so main thing is checking
- that RELOC stuff works, and that bytes and words are correctly
- assembled into the incore array for FASL, passed out to the file
- and correctly re-written. <<<examples of what this looks like?>>>
- FASLOUT and the FASLIN a few small files
- <<<how's about some pre-built tests?>>> to check accuracy. These
- files should be self-contained, and not intially contain
- SYSLISP code, since the SYSLISP.B module has not been built.
- <<< easier said than done- syslisp has had a tendenacy to creep into
- nearly everything for "effeciency" sake...>>>
- For example, try the PU:POLY.RED. An important one
- is PU:RLISP-PARSER.RED and PU:RLISP-SUPPORT.RED.
- [It is worth while to use a small BINARY-DUMP
- routine that reads a binary file and prints it as OCTAL or HEX numbers.
- This can be compared with the known FASL format<<<which is ____>>>,
- for a test file that
- has been fasled on a similar machine].
- Common problems encountered at this phase are:
- @begin(description)
- ???? <<<amen>>>
- @end(description)
- @subsection(FASLOUT the critical files)
- In order to build most of the .B files that are needed, one needs to
- create the IF-SYSTEM, BUILD, RLISP, COMPILER, FASLOUT and LAP modules.
- First "hand-build" the IF-SYSTEM and SYSLISP and BUILD modules:
- @begin(verbatim)
- FASLOUT "IF-SYSTEM";
- IN "IF-SYSTEM.RED"$
- FASLEND;
- @End(verbatim)
- Building SYSLISP is tricker since it needs
- a version of SYSLISP to build from. First edit the PC:SYSLISP.BUILD file,
- to make sure that the IF_SYSTEM clauses mention your machine
- (as set up in the SYSTEM_LIST!* list before). Then read in the
- SYSLISP support interpretively, and then FASLOUT :
- @begin(verbatim)
- <<<where are we? is this with the cross compiler?>>>
- LOAD IF!-SYSTEM; % Needs IF-SYSTEM
- IN "SYSLISP.BUILD"; % To get interpreted SYSLISP in
- % since it needs SYSLISP to build
- OPTIONS!* := 'SYSLISP . OPTIONS!*;
- % To prevent PSL from attempting to load Syslisp;
- FASLOUT "SYSLISP";
- IN "SYSLISP.BUILD"$ % may have to use PATHIN off PC:
- FASLEND;
- @end(verbatim)
- Finally, faslout the BUILD.B module, for future module building:
- @begin(verbatim)
- FASLOUT "BUILD";
- IN "BUILD.BUILD"$
- @end(verbatim)
- Now use BUILD on the other modules that are needed to produce
- the base system:
- @BEGIN(verbatim)
- BUILD 'RLISP;
- BUILD 'COMP!-DECLS;
- BUILD 'PASS!-1!-LAP;
- BUILD 'xxx!-LAP;
- BUILD 'xxx!-CMAC;
- BUILD 'xxx!-COMP;
- BUILD 'FASLOUT;
- BUILD 'COMPILER;
- @end(verbatim)
- @subsection(Test FASL'd RLISP and COMPILER)
- LOAD the RLISP modules into the BARE-PSL
- system, check that RLISP works on a number of files.
- Now LOAD the COMPILER, try some in-core compilation of simple
- procedures (ON COMP).
- Finally use this system to FASLOUT or BUILD a variety of modules.
- Ultimately try rebuilding RLISP and COMPILER and SYSLISP.
- <<<what are problems here? what's the roles of the resident system and the
- cross compiler at this point?>>>
- @subsection(BUILD rest of library)
- Now go through the PU: directory, running BUILD on each of the BUILD
- files. Check each build-file to see which additional modules are needed.
- Important shared modules are:
- @begin(verbatim)
- <<<gee, if you squint this looks like a unix makefile...>>>
- INUM Needs SYSLISP
- FAST-VECTOR Needs SYSLISP, IF-SYSTEM
- VECTOR-FIX Needs SYSLISP
- GSORT Needs SYSLISP
- BIGBIG Needs SYSLISP, FAST-VECTOR,VECTOR-FIX,ARITH
- BIGFACE Needs SYSLISP, FAST-VECTOR,VECTOR-FIX,ARITH
- INUM, BIGBIG,IF-SYSTEM
- @end(verbatim)
- -------
|