guix-package-manager-tutorials/guide.org

Reproducible environments and profiles

How to install previous package versions

Writing a manifest.scm file as follows in itself does not make a basis for a reproducible environment or profile:

#+begin_src scheme (specifications->manifest '("python@3.8.2" "python-redis@3.3.8")) #+end_src

The problem with this approach is, that GNU Guix will not be able to find these specific versions of the packages, when GNU Guix is updated itself. GNU Guix itself is version controlled and with it its references to package sources. The commit of GNU Guix itself needs to be referenced, because different versions of guix contain references to different versions of a package, dropping older versions of packages. It will only know the versions of packages, which come with its precise release. When you update GNU Guix, it will know different versions of packages. It will give an error like the following:

#+begin_quote guix environment: error: python-redis: package not found for version 3.3.8 #+end_quote

The same happens, when you try to make an ad-hoc environment as follows:

#+begin_src shell guix environment --ad-hoc 'python@3.8' 'python-redis@3.3.8' -- python3.8 #+end_src

One could argue, that you should always use the latest version of packages, but that does not make for a reproducible environment.

To make a truly reproducible environment or profile, which "stands the test of time", one needs to do it differently.

To track one specific commit of GNU Guix, one needs to create a channels.scm file. This is done as follows:

#+begin_src shell guix describe --format=channels > channels.scm #+end_src

This will write down a definition inside channels.scm, which specifies exactly, which commit id of GNU Guix to track. Such a definition can be used by the time-machine functionality of GNU Guix in combination with a manifest.scm. Such a manifest.scm could look as follows:

#+begin_src scheme (specifications->manifest '("python" "python-redis")) #+end_src

Note, that there are no versions specified any longer, as they are already determined by the commit, which GNU Guix shall be on, which is specified in the channels.scm.

Installing packages from multiple different channels

Tracking only one channel limits package version choice to that one commit of GNU Guix. It is desirable to be able to use package versions of multiple GNU Guix commits.

A channels.scm file contains a list of channels. For example:

(list (channel (name 'guix) (url "https://git.savannah.gnu.org/git/guix.git") (commit "29a2eb36ff") (introduction (make-channel-introduction "9edb3f66fd" (openpgp-fingerprint "BBB0 2DDF 2CEA F6A8 0D1D E643 A2A0 6DF2 A33A 54FA")))))

In this example only one channel is specified in the list. However, it might work to put multiple channels in there, which have versions of packages, which otherwise never appear in the same version of GNU Guix. (TODO: Test this with some example.) See also https://guix.gnu.org/manual/en/html_node/Channels.html under "Replicating Guix", where an example is given using multiple channels:

;; Deploy specific commits of my channels of interest. (list (channel (name 'guix) (url "https://git.savannah.gnu.org/git/guix.git") (commit "d894ab8e9b")) (channel (name 'my-personal-packages) (url "https://example.org/personal-packages.git") (commit "dd3df5e2c8")))

Although this refers to two different Guix repositories. (TODO: Test whether it works for 2 different repositories, if it does not work with 2 channels for only 1 repository.)

Finding a GNU Guix commit id for packages in a specific version

Currently (at timestamp 2020-10-25) there is no comfortable command line way to do this. To find a commit id of GNU Guix follow the following steps:

1. visit https://hpc.guix.info/browse
2. enter the package name, for example python-redis
3. click the package name in the results of the search, for example https://hpc.guix.info/package/python-redis
4. click on "View package version history", for example leading to http://data.guix.gnu.org/repository/1/branch/master/package/python-redis

on the page shown you can see the versions of the package you searched. for example for python-redis one can see: 2.10.6, 3.2.0, 3.2.1, 3.3.8, and 3.5.3 (at timestamp 2020-10-25). for each version the earliest and latest date when GNU Guix had this version is shown.

1. click on the latest date of for the version you want

the dates link actually to a page displaying information about GNU Guix at that time, which means at one specific commit. from this page you can extract the so called "revision".

1. use the revision to specify a Guix commit using the following command:

#+begin_src shell guix time-machine --commit=a95057ccee -- environment --ad-hoc python@3.8.2 python-redis@3.5.3 -- python3.8 #+end_src

This should be a reproducible environment and work wherever GNU Guix and the required build tools are available.

You can also put that commit of Guix in a channels.scm file by using the following command:

guix time-machine --commit=a95057ccee -- describe --format=channels > channels.scm

The Guix time machine command is very elegant in its usage, letting you simply specify a command to run with a specific version of Guix by appending -- <other command> <arguments of other command>.

Calculate Guix hash

for tarballs

guix hash the-file

for repositories

guix hash --exclude-vcs --recursive .

Importing packages

PyPI

Versions numbers are specified as follows:

guix import pypi jupyterlab_server/2.0.0rc1

Other importers might use different characters to delimit versions.

Native inputs vs propagated inputs

For python packages it doesn't matter at build time, it's more about if it's needed at build time (native-inputs) or if it's needed to run the python program (propagated-inputs) since we don't have a good way to tell python to keep a reference to other python packages.

Debugging

Looking at logs

Assuming WorkspacesAPITest does actually have an attribute named lab_config then you'll want to add the installed package to the python path during the check phase. There are a number of packages which do this. I can never remember the syntax so I always grep for 'add-installed-pythonpath' when it comes up.

You can use the following script to look at logs:

#!/usr/bin/env bash

set -Eeuxo pipefail

DIR="$(cd "$(dirname "$0")" > /dev/null 2>&1; pwd -P)" printf "script directory: %s\n" "${DIR}"

if [ -z ${1+x} ]; then printf "%s\n" "No log file specified."; exit 1; else printf "%s %s %s\n" '${1}' 'is set to' "${1}"; COMPRESSED_LOG_FILE="${1}" fi

COMPRESSED_FILE_NAME="$(basename -- ${COMPRESSED_LOG_FILE})"

DECOMPRESSED_FILE_NAME="${COMPRESSED_FILE_NAME%.*}"

cp --verbose "${COMPRESSED_LOG_FILE}" "${DIR}/${COMPRESSED_FILE_NAME}"

bzip2 --decompress "${COMPRESSED_FILE_NAME}"

cat "${DECOMPRESSED_FILE_NAME}" rm --verbose "${DECOMPRESSED_FILE_NAME}"