reduce-algebra/README.BUILDING

                       Fetching and Building REDUCE
                       ============================

                               January 2015
                                 June 2020



The notes here are about having a full set of Reduce sources fetched from
Sourceforge and compiling them on your own computer. Following that path
lets you track the most recent changes, updates and corrections. Because
both Reduce and the platforms that people may wish to build on change over
time what is explained here will almost always have a few details where it
is out of date: so check the date to see how recently this has been refreshed.
But almost always the changes needed to what is explained here will be small,
and if you find some please report back to the Reduce mailing list for the
benefit of others.

The prerequisites for using this package will depend somewhat on whether
you choose to use the CSL or PSL build.

Users of PSL will use the ready-build binaries of the PSL Lisp system that
are present in the material hosted at Sourceforge. Rebuilding PSL from source
is possible but the procedures are not covered at all here and anybody needing
to do that probably needs to consult the PSL experts. Because PSL uses
pre-made binaries these have to be present for the architecture you are using.
They are available for all the most important or common cases. Because the
binaries are pre-made there are few demands for compilers or special
libraries installed on the computer you will use PSL on.

For CSL you need to be able to compile both CSL itself and the FOX toolkit
on which it depends. Please note that even if you have FOX available already
on your computer the code her uses a (slightly) customised version and your
standard system-wide installation of FOX will not be used and should be
irrelevant here. To join in development work on what is hoped to be a future
version of CSL-Reduce you would also need to be able to work with wxWidgets.

You will need:
   For Windows, the "cygwin" environment providing a "bash" shell.
       Note that building under "msys" or the plain Windows command
       prompt is not possible (without you doing significant work of your
       own to arrange for it). Obviously it will be sensible to keep your
       cygwin installation up to date, and some updates there have in the
       past not always been upards compatible. The aim is to keep the
       current CSL/Reduce at Sourceforge buildable using the most recent
       cygwin release. But any difficulty due to temporary bugs or glitches
       at the cygwin end can not be resolved here! These days you MUST use
       the 64-bit version of cygwin - note that the obsolete 32-bit one is
       deprecated by its suppliers.
       The command scripts/cygwin-sanity-check.sh attempts to check if
       sufficient cygwin components have been installed, and it may save
       you pain if you try it and respond to its suggestions before trying
       a full build of Reduce.
   For Macintosh you need the XCode command line tools, and you are liable
       to need to use Macports to fetch and install a significant number
       of commands and libraries that Apple do not provide as standard.
       There is a file in the scripts directory that I use to fetch the
       set of "ports" that I use. You will also need Xquartz to support the
       GUI.
   For Linux you can inspect scripts/ubuntu-sanity-check.sh or the fedora
       equivalent to find a script that installs what is probably a superset
       of the packages you will need.
   For other platforms there is a hope that everything will work if you
       can provide a bash shell and enough development libraries, but you
       mat need to sort out details for yourself. Here are a few more comments:
   GNU make (other versions of "make" are not liable to work), autoconf,
       automake and libtool. All should be at least tolerably up to date
       versions.
   You need a C++ development context. On Windows you need the
       and "mingw-x86_64" compiler variants (as well as plain g++) from
       cygwin while on other systems you need X11 and other
       development capability, including Xft, fontconfig, (n)curses, Posix
       threads. The building has been tested using versions of both gcc
       and clang. The compiler should be new enough that it supports C++11
       and for choice C++17.

I have (recently) built using cygwin64, Ubuntu and Fedora Linuxes
on pcs, Raspbian (raspberry Pi OS, both 32 and 64-bit variants) on a
Raspberry Pi and OSX Catalina. In the past I have built using PCBSD/FreeBSD
and Solaris on both pc and sparc (requiring significant
extra software installation), SGI, HP-pa, the IBM z-architecture and at one
stage (not now I think) I coudd run Reduce on an HP IPAQ 4700 PDA or an
openWRT router. The hope is that almost any machine with a C++ compiler
should suffice for the basic system. In case of a challenging environment
it may be prudent to do initial testing by running
"./configure --with-csl --without-gui" to avoid all attempts to build
windowing libraries. Also in recent years I have started to suppose that all
computers have substantial amounts of memory, so ones with only a few megabytes
(or even only tend or hundreds) may cause problems.

One of the most common problems people seem to have the first time that they
try to build CSL arises when they do not have quite all the necessary tools,
libraries and header files installed. While most relevant run-time libraries
are liable to have been provided for them by default, the development versions
might not have. The state of your cygwin installation can be checked using
scripts/cygwin-sanity-check.sh but please be aware that the tests that does
are indicative rather than definitive and when (occasionally) either Reduce
is extended and additional dependencies arise or cygwin is reorganised and the
names of some packages alter the tests may become outdated.

If you try to build REDUCE but one of the programming tools or development
libraries that is required is not available then there is no guarantee that
the failure that results will describe itself to you clearly! If the failure
is detected at configuer time the evidence will be in a file "config.log"
(possibly somewhere within the cslbuild directory). In case of trouble I often
go
    script buildlog.log
    rm -rf cslbuild
    ./configure --with-csl
    make
    exit
and then buildlog.log contains a full transcript of all the output from the
attempt. Its size can be daunting and distinguishing between messages,
warnings, observations and errors can take experience!

To try to make that easier to cope with I provide a script that attempts to
check for prerequisites and that is lighter weight than the full configuration
process and maybe reports trouble in a cleaner way. To use it select the
Reduce "trunk" directory as current and at a bash prompt (you MUST have
"bash" available as a shell, and you will run least risk if you run all
scripts from it. I have tried to make most of them compatible with a legacy
/bin/sh but testing and guaranteeing that is tedious) go
      scripts/csl-sanity-check.sh
This should run auto-configure scripts and eventually compile and run a small
program that pops up a dull window. If you get the window at the end it has
succeeded and you can try building Reduce. If not then it probably reports
a step where some header file or library is missing. You need to search the
installation sources relevant your operating system and install more stuff,
then try again. Eg for cygwin you re-run "setup", while on Linux you are
looking at "apt-get" and "yum" and various other package managers. You need
to check the documentation of your particular operating system - or perhaps
better find local expert to get you started.

Note that if you configure using "--with-csl --with-wx" you also need GTK2
(GTK+) development files if you are on an X11-based platform, but at present
csl-sanity-check.sh does not test for that.

Some people may find that they need things that are not installed on
their system but they have not been granted adminitrator authority and so
can not merely install them in the obvious way. The easiest response is to
talk to a local system manager! But for a range of the tools (eg autoconf,
make, ncurses etc) if you can not identify a ready-built binary
package that suits your environment it will be easy to build one from sources
found "at the usual places", and may be easy to arrange that the resulting
binaries are on your PATH and libraries somewhere where they will be found.
But in such a case you may end up needing to hand-patch some of the Makefiles
here.

If you have run the main CSL/Reduce configure step, as in
   ./configure --with-csl
and tried "make" at a stage when you do not have all relevant headers and
libraries installed then the build can fail part way through in a messy way.
In principle you can check ALL the logs and work out what happened, but in
reality that can be daunting. Furthermore it can get in a state where a
subsequent use of "make" fails in an unhelpful manner. Experts can patch
things up and recover, but the simplest approach in case of confusion is
   rm -rf cslbuild
   ./configure --with-csl
   make
where all the partially build stuff that had accumulated in cslbuild is
discarded and the configuration and compilation is started afresh. Obviously
you will not delete that directory if you have put important files of your
own there!

A feature of subversion is that if you change any file locally then your local
edit is interpreted as a possibly valuable update that you may at some stage
contribute back into the project. This is good, but it means that if you
accidentally corrupt a local copy of some file then subversion will NOT
automatically repair it. So if you have just recently started with Reduce and
something odd is happening it may make sense to get yourself back in lock-step
with the central repository. You are provided with two scripts
   scripts/make-pristine.sh
   scripts/stamp.sh
The first of those merely issues the subversion commands that discard all local
changes you have made to bring everything into step. The second updates the
time-stamps on various configuration files so that when you run "configure"
it should not attempt to use autoconf and automake locally, but instead should
be content with the files as fetched. These two scripts can avoid confusion
early on. Once you are fully working and especially if you are doing your own
development within Reduce you will not wish to use them.


You can fetch or update REDUCE in one of (at least) two ways

(1) Fetch a file from the Sourceforge download area, eg from the
reduce-algebra-source section. Unpack it. Note that the full archives
at sourceforge are updated sporadically and not to any coherent schedule,
so what you get could be a little out of date. Thus this scheme is not
recommended!

OR

(2) Use subversion to fetch or refresh a copy from the master site.
        svn checkout \
           svn://svn.code.sf.net/p/reduce-algebra/code/trunk \
           reduce-algebra
    That should create a directory called "reduce-algebra" for you.
    To refresh your files, enter the reduce-algebra directory and go
        svn update
    You can check the documentation of subversion (svn) to discover
    what further options and possibilities tare provided. Two notable
    advanntages of this are (a) "svn update" lets you refresh your copy
    to match the very latest one available without needing to fetch
    everything again afresh and (n) "svn -r NNNN update" lets you update
    you local copy to match "revision NNNN", a possibly older copy of
    everything, so if the main development version is even temporarily
    broken (that should only happen accidentally!) you can fetch older
    versions for comparison.

The version fetched and updated using subversion (hah - note that you
need "subversion" installed on your computer to do this!) will be fully
up to date. That naturally means that at some moments it will include
changes that temporarily break it, but in general any such should be
corrected fairly rapdily. Furthermore by browsing the documentation of
subversion you can find out how to check out a version from say 2 weeks
ago (or indeed any historical varient) if needbe.



The simplest way version of the build process is to select the directory
where REDUCE was unpacked as current and issue one of the command
         ./configure --with-csl
     OR  ./configure --with-psl
(or first one of those then the other)

It ought not to be necessary to provide any other options to the configure
script or to pre-set environment variables, but for experts there are (of
course) many opportunities for customisation. If you find you are on a platform
where simple use of the configure script is insufficient please report that
so that things can be updated and users in the future will come closer to
find that everything works first time without fuss.

Follow that by the command "make" and if all goes well everything relevant
will be compiled. But at least in the CSL case please try
scripts/csl-sanity-check.sh first - it may save you a lot of agony.

If you build both the CSL and PSL versions then it can be a comfort to
run
    scripts/testall.sh
which runs all the various test scripts and displays a report showing any
places where the output on your machine does not match reference versions
of the logs. Directories called csl-times and psl-times end up with the
log output. In some cases log discrepancies are expected! For instance some
of the test files generate timing reports so their output varies depending on
the speed or size of your computer. In other cases improvements to the
system may change test output but the reference logs may not have been
updated to reflect it. Some logs are there to highlight difference between
detailed behaviour of CSL and PSL that are not hugely important but that
may be addressed at some stage in the future.

For (some sort of) convenience a script "rebuild.sh" in the top level of
the distribution tries to do this for both PSL and CSL and it then leaves
overall log files also at the top level.

You can then run REDUCE using the command
    bin/redcsl
or
    bin/redpsl

[At present I do not in any way guarantee (and I probably do not even expect)
that the usual "configure" option "--prefix=..." or the idiom "make install"
will behave as you might have hoped. Indeed there are reports that at
present "make install" is deeply broken and SHOULD NOT BE ATTEMPTED.
But there is scripts/mk-snapshots.sh which is used to build the distribution
snapshots and those can then be installed.]

Any machine you are on is attributed an identity such as x86_64-pc-windows
or x86_64-unknown-ubuntu10.10. Note that this is slightly more specific than
the normal GNU host triple in that it attempts to include the name of a Linux
distribution. Binaries are actually built within cslbuild/<triple> or
pslbuild/<triple>. This arrangement is intended to be helpful if you have one
file-space shared between several machine that may have different
architectures. So the scripts like bin/redcsl deduce where to look and
chain to binaries there. In the CSL case you could make a symbolic link
as in "ln -s .../trunk/cslbuild/<triple>/csl/reduce XXXX" where XXXX is some
location on your PATH which then lets you launch Reduce more directly. Reduce
needs some resources in the same directory as the main executable so merely
copying <triple>/csl/reduce to somewhere else is not liable to be a good idea.

Logs from building and testing Reduce end up in the directory where the
main binary gets created, so if things go wrong that is where you may wish
to look.



There are a number of optional ways in which configuration may be customised:
         ./configure --with-csl --enable-debug
   Enable "gcc -g" compilation of the C parts of CSL so that gdb can be
   used to debug things.

         ./configure --with-csl --without-gui
         ./configure --with-csl --with-wx
   By default the CSL version build in support for a GUI using the FOX
   toolkit. If for some reason that is not required or can not be supported
   then "--without-gui" removes it. There are some plans to change to use
   wxWidgets rather than FOX, and "--with-wx" enables the experimental build
   for that. At the time of writing the code compiles but is significantly
   unfinished. Volunteers to help complete it would be useful! But neither of
   these options are a very good idea for ordinary users!

   There are other options, many of which are experimental or historical.
   See "./configure --help" for a definitive list.


Developers and some more ambitious users may need to understand the
directory layout in more detail. The document "Inside Reduce" the
the "primers" part of the Reduce documentation directory may provide
useful guidance.
The main directories supplied are as follows, where MACHINE stands for
a machine identification, eg x86_64-pc-windows-debug.

 trunk                    The main version of everything
   bin                    Scripts to launch REDUCE etc
   csl/cslbase            Main CSL Lisp sources
       fox                the FOX GUI toolkit, an LGPL package
   cslbuild/generated-c   Part of REDUCE compiled into C
            MACHINE/csl   where REDUCE binaries are built
                    fox   where the FOX GUI toolkit is built
   packages/alg           REDUCE sources, on package per directory
            ...
            ztrans
   psl/psl-amd64          Ready-build binaries of the PSL Lisp system ...
       ...                ...
       psl/win32          ... for a variety of architectures
   pslbuild/MACHINE/.     where REDUCE gets built
                    psl   copy of relevant architecture-specific PSL
                    red   REDUCE built binary "fasl" and image files
                    deps  used to record dependencies in the build
   scripts                scripts used to maintain the system.



If you are working on just one platform and have completed the configuration
and done an initial build you may like to select either cslbuild/MACHINE/csl
or pslbuild/MACHINE as your current directory: the Makefile that ends up there
is the main one used for rebuilding the system.


For special purposes you may wish to investigate the csl/jlisp directory (all
Reduce runnable via Java - distinctly slower and at present the user interface
is not well sorted, but it might be fun) and csl/new-embedded (a kit for
making a simple small C-coded version for potential use as a component in
some other software rather than a free-standing complete program). Both of
these are for experienced programmers who want to use them in some broader
project rather than being fully supported variants on Reduce itself.

There are also "spins" of Reduce by way of extra files and scripts that have
made it possible to build the system on other Lisps, from Common Lisp as
far as emacs lisp. There are a range of alternative interfaces for both
console and GUI purposes in varying states of support and maturity.


The more alive the Reduce project is the more you can expect there to be
additions, changes and enhancements but please understand that these all
carry risk of transient introduction of bugs. Report trouble to us via
Sourceforge and when and if we can we will fix them. But all the source code
is there so you can also join in the debugging and propose fixes (please!)



A C Norman. July 2008. Updated Jan 2009, September 2010, March 2011,
            September 2014, January 2015, June 2020.