dragora-handbook/en/qi-content.texi

@c -*-texinfo-*-
@c qi-content.texi
@c This is part of the Qi user guide.
@c Copyright (C) 2019-2022 Matias Fonzo, <selk@dragora.org>.
@c See the file qi-header.texi for copying conditions.
@c
@c If something is modified here, be aware of the increase of VERSION in
@c qi-header.texi, this will produce the result of the manual: qi.info

@node Introduction to Qi
@chapter Introduction to Qi
@cindex introduction to qi

Qi is a simple but well-integrated package manager.  It can create,
install, remove, and upgrade software packages.  Qi produces binary
packages using recipes, which are files containing specific instructions
to build each package from source.  Qi can manage multiple packages
under a single directory hierarchy.  This method allows to maintain a set
of packages and multiple versions of them.  This means that Qi could be
used as the main package manager or complement the existing one.

Qi offers a friendly command line interface, a global configuration
file, a simple recipe layout to deploy software packages; also works
with binary packages in parallel, speeding up installations and packages
in production.  The format used for packages is a simplified and safer
variant of POSIX pax archive compressed in lzip format.

Qi is a modern (POSIX-compliant) shell script released under the
terms of the GNU General Public License.  There are only two major
dependencies for the magic: graft(1) and tarlz(1), the rest is expected
to be found in any Unix-like system.

@node Invoking qi
@chapter Invoking qi
@cindex invocation

This chapter describes the synopsis for invoking Qi.

@example
Usage: qi COMMAND [@var{OPTION}...] [@var{FILE}]...
@end example

@noindent
One mandatory command specifies the operation that @samp{qi} should
perform, options are meant to detail how this operation should be
performed during or after the process.

@noindent
Qi supports the following commands:

@table @code
@item warn
Warn about files that will be installed.

@item install
Install packages.

@item remove
Remove packages.

@item upgrade
Upgrade packages.

@item extract
Extract packages for debugging purposes.

@item create
Create a .tlz package from directory.

@item build
Build packages using recipe names.

@item order
Resolve build order through .order files
@end table

@noindent
Options when installing, removing, or upgrading software packages:

@table @code
@item -f
@itemx --force
Force upgrade of pre-existing packages.

@item -k
@itemx --keep
Keep directories when build/remove/upgrade.

Keep (don't delete) the package directory when using remove/upgrade command.

This will also try to preserve the directories @samp{$@{srcdir@}} and
@samp{$@{destdir@}} when using build command.  Its effect is available in
recipes as @samp{$@{keep_srcdir@}} and @samp{$@{keep_destdir@}}.  See
@ref{Recipes, Special variables} for details.

@item -p
@itemx --prune
Prune conflicts.

@item -P
@itemx --packagedir=<dir>
Set directory for package installations.

@item -t
@itemx --targetdir=<dir>
Set target directory for symbolic links.

@item -r
@itemx --rootdir=<dir>
Use the fully qualified named directory as the root directory for all qi
operations.

Note: the target directory and the package directory will be
relative to the specified directory, excepting the graft log file.
@end table

@noindent
Options when building software packages using recipes:

@table @code
@item -a
@itemx --architecture
Set architecture name for the package.

@item -j
@itemx --jobs
Parallel jobs for the compiler.

This option sets the variable @samp{$@{jobs@}}.  If not specified, default
sets to 1.

@item -S
@itemx --skip-questions
Skip questions on completed recipes.

@item -1
@itemx --increment
Increment release number (@samp{$@{release@}} + 1).

The effect of this option will be omitted if --no-package is being used.

@item -n
@itemx --no-package
Do not create a .tlz package.

@item -i
@itemx --install
Install package after the build.

@item -u
@itemx --upgrade
Upgrade package after the build.

@item -o
@itemx --outdir=<dir>
Where the packages produced will be written.

This option sets the variable @samp{$@{outdir@}}.

@item -w
@itemx --worktree=<dir>
Where archives, patches, recipes are expected.

This option sets the variable @samp{$@{worktree@}}.

@item -s
@itemx --sourcedir=<dir>
Where compressed sources will be found.

This option sets the variable @samp{$@{tardir@}}.
@end table

@noindent
Other options:

@table @code
@item -v
@itemx --verbose
Be verbose (an extra -v gives more).

It sets the verbosity level, default sets to 0.

The value 1 is used for more verbosity while the value 2 is too detailed.
Although at the moment it is limited to graft(1) verbosity.

@item -N
@itemx --no-rc
Do not read the configuration file.

This will ignore reading the qirc file.

@item -L
@itemx --show-location
Print default directory locations and exit.

This will print the target directory, package directory, working tree,
the directory for sources, and the output directory for the packages
produced.  The output will appear on STDOUT as follows:

@example
QI_TARGETDIR=/usr/local
QI_PACKAGEDIR=/usr/local/pkgs
QI_WORKTREE=/usr/src/qi
QI_TARDIR=/usr/src/qi/sources
QI_OUTDIR=/var/cache/qi/packages
@end example

You can set these environment variables using one of the following methods:

@code{eval "$(qi -L)"}

This will display the default locations taking into account the values set
from the qirc configuration file.  You can deny the influence of the
configuration file by setting the option @samp{-N}.

@code{eval "$(qi -N -L)"}

Or you can adjust the new locations using the command-line options, e.g:

@code{eval "$(qi -N --targetdir=/directory -L)"}

@item -h
@itemx --help
Display the usage and exit.

@item -V
@itemx --version

This will print the (short) version information and then exit.

The same can be achieved if Qi is invoked as @samp{qi version}.
@end table

When FILE is -, qi can read from the standard input.  See examples from
the @ref{Packages} section.

Exit status: 0 for a normal exit, 1 for minor common errors (help usage,
support not available, etc), 2 to indicate a command execution error;
3 for integrity check error on compressed files, 4 for empty, not
regular, or expected files, 5 for empty or not defined variables,
6 when a package already exist, 10 for network manager errors.
For more details, see the @ref{Qi exit status} section.


@node The qirc file
@chapter The qirc file
@cindex configuration file

The global @file{qirc} file offers a way to define variables and tools
(such as a download manager) for default use.  This file is used by qi
at runtime, e.g., to build, install, remove or upgrade packages.

Variables and their possible values must be declared as any other
variable in the shell.

@noindent
The command line options related to the package directory and target
directory and some of the command line options used for the build command,
have the power to override the values declared on @file{qirc}.
See @ref{Invoking qi}.

@noindent
The order in which qi looks for this file is:

@enumerate
@item
@env{$@{HOME@}/.qirc}
@- Effective user.

@item
@samp{$@{sysconfdir@}/qirc}
@- System-wide.
@end enumerate

If you intend to run qi as effective user, the file
@samp{$@{sysconfdir@}/qirc} could be copied to @env{$@{HOME@}/.qirc}
setting the paths for @samp{$@{packagedir@}} and @samp{$@{targetdir@}}
according to the @env{$HOME}.


@node Packages
@chapter Packages
@cindex managing packages

A package is a suite of programs usually distributed in binary form
which may also contain manual pages, documentation, or any other file
associated to a specific software.

The package format used by qi is a simplified POSIX pax archive
compressed using lzip@footnote{For more details about tarlz and the
lzip format, visit @url{https://lzip.nongnu.org/tarlz.html}.}.  The
file extension for packages ends in @samp{.tlz}.

@noindent
Both package installation and package de-installation are managed using
two important (internal) variables: @samp{$@{packagedir@}} and
@samp{$@{targetdir@}}, these values can be changed in the
configuration file or via options.

@samp{$@{packagedir@}} is a common directory tree where the package
contents will be decompressed (will reside).

@samp{$@{targetdir@}} is a target directory where the links will be
made by graft(1) taking @samp{$@{packagedir@}/package_name} into account.

@noindent
Packages are installed in self-contained directory trees and symbolic
links from a common area are made to the package files.  This allows
multiple versions of the same package to coexist on the same system.

@section Package conflicts
@cindex package conflicts

All the links to install or remove a package are handled by graft(1).
Since multiple packages can be installed or removed at the same time,
certain conflicts may arise between the packages.

@noindent
graft@footnote{The official guide for Graft can be found at
@url{https://peters.gormand.com.au/Home/tools/graft/graft.html}.}
defines a CONFLICT as one of the following conditions:

@itemize @bullet
@item
If the package object is a directory and the target object exists but is
not a directory.

@item
If the package object is not a directory and the target object exists
and is not a symbolic link.

@item
If the package object is not a directory and the target object exists
and is a symbolic link to something other than the package object.
@end itemize

@noindent
The default behavior of qi for an incoming package is to ABORT if a
conflict arises.  When a package is going to be deleted, qi tells to
graft(1) to remove those parts that are not in conflict, leaving the
links to the belonging package.  This behavior can be forced if the
--prune option is given.

@section Installing packages
@cindex package installation

To install a single package, simply type:

@example
qi install coreutils_8.30_i586-1@@tools.tlz
@end example

@noindent
To install multiple packages at once, type:

@example
qi install gcc_8.3.0_i586-1@@devel.tlz rafaela_2.2_i586-1@@legacy.tlz ...
@end example

@noindent
Warn about the files that will be linked:

@example
qi warn bash_5.0_i586-1@@shells.tlz
@end example

This is to verify the content of a package before installing it.

@noindent
See the process of an installation:

@example
qi install --verbose mariana_3.0_i586-1@@woman.tlz
@end example

A second --verbose or -v option gives more (very verbose).

@noindent
Installing package in a different location:

@example
qi install --rootdir=/media/floppy lzip_1.21_i586-1@@compressors.tlz
@end example

Important: the --rootdir option assumes @samp{$@{targetdir@}} and
@samp{$@{packagedir@}}.  See the following example:

@example
qi install --rootdir=/home/selk lzip_1.21_i586-1@@compressors.tlz
@end example

The content of "lzip_1.21_i586-1@@compressors.tlz" will be decompressed
into @samp{/home/selk/pkgs/lzip_1.21_i586-1@@compressors}.
Assuming that the main binary for lzip is under
@samp{/home/selk/pkgs/lzip_1.21_i586-1@@compressors/usr/bin/}
the target for "usr/bin" will be created at @samp{/home/selk}.  Considering
that you have exported the @env{PATH} as @samp{$@{HOME@}/usr/bin}, now the
system is able to see the recent lzip command.

@noindent
Installing from a list of packages using standard input:

@example
qi install - < PACKAGELIST.txt
@end example

Or in combination with another tool:
@example
sort -u PACKAGELIST.txt | qi install -
@end example

The sort command will read and sorts the list of declared packages,
while trying to have unique entries for each statement.  The output
produced is captured by Qi to install each package.

An example of a list containing package names is:
@example
/var/cache/qi/packages/amd64/tcl_8.6.9_amd64-1@@devel.tlz
/var/cache/qi/packages/amd64/tk_8.6.9.1_amd64-1@@devel.tlz
/var/cache/qi/packages/amd64/vala_0.42.3_amd64-1@@devel.tlz
@end example

@section Removing packages
@cindex package de-installation

To remove a package, simply type:

@example
qi remove xz_5.2.4_i586-1@@compressors.tlz
@end example

@noindent
Remove command will match the package name using @samp{$@{packagedir@}} as
prefix.  For example, if the value of @samp{$@{packagedir@}} has been
set to /usr/pkg, this will be equal to:

@example
qi remove /usr/pkg/xz_5.2.4_i586-1@@compressors
@end example

@noindent
Detailed output:

@example
qi remove --verbose /usr/pkg/xz_5.2.4_i586-1@@compressors
@end example

A second --verbose or -v option gives more (very verbose).

@noindent
By default the remove command does not preserve a package directory after
removing its links from @samp{$@{targetdir@}}, but this behavior can be
changed if the --keep option is passed:

@example
qi remove --keep /usr/pkg/lzip_1.21_i586-1@@compressors
@end example

This means that the links to the package can be reactivated, later:

@example
cd /usr/pkg && graft -i lzip_1.21_i586-1@@compressors
@end example

@noindent
Removing package from a different location:

@example
qi remove --rootdir=/home/cthulhu xz_5.2.4_i586-1@@compressors
@end example

@noindent
Removing a package using standard input:

@example
echo vala_0.42.3_amd64-1@@devel | qi remove -
@end example

This will match with the package directory.

@section Upgrading packages
@cindex package upgrade

The upgrade command inherits the properties of the installation and removal
process.  To make sure that a package is updated, the package is installed
in a temporary directory taking @samp{$@{packagedir@}} into account.  Once
the incoming package is pre-installed, qi can proceed to search and delete
packages that have the same name (considered as previous ones).  Finally,
the package is re-installed at its final location and the temporary
directory is removed.

Since updating a package can be crucial and so to perform a successful
upgrade, from start to finish, you will want to ignore some important
system signals during the upgrade process, those signals are SIGHUP,
SIGINT, SIGQUIT, SIGABRT, and SIGTERM.

@noindent
To upgrade a package, just type:

@example
qi upgrade gcc_9.0.1_i586-1@@devel.tlz
@end example

This will proceed to upgrade "gcc_9.0.1_i586-1@@devel" removing any other
version of "gcc" (if any).

@noindent
If you want to keep the package directories of versions found during the
upgrade process, just pass:

@example
qi upgrade --keep gcc_9.0.1_i586-1@@devel.tlz
@end example

@noindent
To see the upgrade process:

@example
qi upgrade --verbose gcc_9.0.1_i586-1@@devel.tlz
@end example

A second --verbose or -v option gives more (very verbose).

@noindent
To force the upgrade of an existing package:

@example
qi upgrade --force gcc_9.0.1_i586-1@@devel.tlz
@end example

@subsection Package blacklist
@cindex package blacklist

To implement general package facilities, either to install, remove or
maintain the hierarchy of packages in a clean manner, qi makes use of the
pruning operation via graft(1) by default:

There is a risk if those are crucial packages for the proper functioning
of the system, because it implies the deactivation of symbolic from the
target directory, @emph{especially} when transitioning an incoming package
into its final location during an upgrade.

@noindent
A blacklist of package names has been devised for the case where
a user decides to upgrade all the packages in the system, or
just the crucial ones, such as the C library.

The blacklist is related to the upgrade command only, consists in installing
a package instead of updating it or removing previous versions of it;
the content of the package will be updated over the existing content at
@samp{$@{packagedir@}}, while the existing links from
@samp{$@{targetdir@}} will be preserved.  A pruning of links will be
carried out in order to re-link possible differences with the recent
content, this helps to avoid leaving dead links in the target directory.

@noindent
Package names for the blacklist to be declared must be set from the
configuration file.  By default, it is declared using the package name,
which is more than enough for critical system packages, but if you want to
be more specific, you can declare a package using:
@samp{$@{pkgname@}_$@{pkgversion@}_$@{arch@}-$@{release@}} where
the package category is avoided for common matching.  See
@ref{Recipes, Special variables} for a description of these variables.

@node Recipes
@chapter Recipes
@cindex recipes

A recipe is a file telling qi what to do.  Most often, the recipe tells
qi how to build a binary package from a source tarball.

A recipe has two parts: a list of variable definitions and a list of
sections.  By convention, the syntax of a section is:

@example
section_name()
@{
    section lines
@}
@end example

The section name is followed by parentheses, one newline and an opening
brace.  The line finishing the section contains just a closing brace.
The section names or the function names currently recognized are
@samp{build}.

The @samp{build} section (or @strong{shell function}) is an augmented
shell script that contains the main instructions to build software
from source.

If there are other functions defined by the packager, Qi detects them
for later execution.

@section Variables
@cindex variables

A "variable" is a @strong{shell variable} defined either in @file{qirc}
or in a recipe to represent a string of text, called the variable's
"value".  These values are substituted by explicit request in the
definitions of other variables or in calls to external commands.

Variables can represent lists of file names, options to pass to
compilers, programs to run, directories to look in for source files,
directories to write output to, or anything else you can imagine.

Definitions of variables in qi have four levels of precedence.
Options which define variables from the command-line override those
specified in the @file{qirc} file, while variables defined in the recipe
override those specified in @file{qirc}, taking priority over those
variables set by command-line options.  Finally, the variables have
default values if they are not defined anywhere.

Options that set variables through the command-line can only reference
variables defined in @file{qirc} and variables with default values.

Definitions of variables in @file{qirc} can only reference variables
previously defined in @file{qirc} and variables with default values.

Definitions of variables in the recipe can only reference variables
set by the command-line, variables previously defined in the recipe,
variables defined in @file{qirc}, and variables with default values.

@section Special variables
@cindex special variables

There are variables which can only be set using the command line options or
via @file{qirc}, there are other special variables which can be defined or
redefined in a recipe.  See the following definitions:

@samp{outdir} is the directory where the packages produced are written.
This variable can be redefined per-recipe.  Default sets to
@samp{/var/cache/qi/packages}.

@samp{worktree} is the working tree where archives, patches, and recipes
are expected.  This variable can not be redefined in the recipe.  Default
sets to @samp{/usr/src/qi}.

@samp{tardir} is defined in the recipe to the directory where the tarball
containing the source can be found.  The full name of the tarball is
composed as @samp{$@{tardir@}/$tarname}.  Its value is available in the
recipe as @samp{$@{tardir@}}; a value of . for @samp{tardir} sets it to
the value of CWD (Current Working Directory), this is where the recipe
lives.

@samp{arch} is the architecture to compose the package name.  Its value is
available in the recipe as @samp{$@{arch@}}.  Default value is the one
that was set in the Qi configuration.

@samp{jobs} is the number of parallel jobs to pass to the compiler.  Its
value is available in the recipe as @samp{$@{jobs@}}.  The default value
is 1.

The two variables @samp{$@{srcdir@}} and @samp{$@{destdir@}} can be
set in the recipe, as any other variable, but if they are not, qi uses
default values for them when building a package.

@samp{srcdir} contains the source code to be compiled, and defaults to
@samp{$@{program@}-$@{version@}}.  @samp{destdir} is the place where the
built package will be installed, and defaults to
@samp{$@{TMPDIR@}/package-$@{program@}}.

If @samp{pkgname} is left undefined, the special variable @samp{program}
is assigned by default.  If @samp{pkgversion} is left undefined, the
special variable @samp{version} is assigned by default.

@samp{pkgname} and @samp{pkgversion} along with: @samp{version}, @samp{arch},
@samp{release}, and (optionally) @samp{pkgcategory} are used to produce the
package name in the form:
@samp{$@{pkgname@}_$@{pkgversion@}_$@{arch@}-$@{release@}[@@$@{pkgcategory@}].tlz}

@samp{pkgcategory} is an optional special variable that can be defined on the
recipe to categorize the package name.  If it is defined, then the
package output will be composed as
@samp{$@{pkgname@}_$@{pkgversion@}_$@{arch@}-$@{release@}[@@$@{pkgcategory@}.tlz}.
Automatically, the value of @samp{pkgcategory} will be prefixed using the
@samp{@@} (at) symbol which will be added to the last part of the package name.

A special variable called @samp{replace} can be used to declare package names
that will be replaced at installation time.

The special variables @samp{keep_srcdir} and @samp{keep_destdir} are provided
in order to preserve the directories @samp{$@{srcdir@}} or @samp{$@{destdir@}},
if those exists as such.  Note: The declaration of these variables are subject
to manual deactivation; its purpose in recipes is to preserve the directories
that relate to the package's build (source) and destination directory, that is
so that another recipe can get a new package (or meta package) from there.  For
example, the declarations can be done as:

@example
keep_srcdir=keep_srcdir
keep_destdir=keep_destdir
@end example

Then from another recipe you would proceed to copy the necessary files that
will compose the meta package, from the main function you must deactivate
the variables at the end:

@example
unset -v keep_srcdir keep_destdir
@end example

This will leave the 'keep_srcdir' and 'keep_destdir' variables blank to
continue with the rest of the recipes.

The special variable @samp{opt_skiprecipe} is available when you need to
ignore a recipe cleanly, continuing with the next recipe.  May you add a
conditional test then set it as @samp{opt_skiprecipe=opt_skiprecipe}.

The variable @samp{tarlz_compression_options} can be used to change the
default compression options in tarlz(1), default sets to @samp{-9 --solid}.
For example if the variable is declared as:

@example
tarlz_compression_options="-0 --bsolid"
@end example

It will change the granularity of tarlz(1) by using the @samp{--bsolid}
option @footnote{About the @samp{--bsolid} granularity option of tarlz(1),
@url{https://www.nongnu.org/lzip/manual/tarlz_manual.html#g_t_002d_002dbsolid}.},
as well as increasing the compression speed by lowering the compression
level with @samp{-0}.

This is only recommended for recipes where testing, or faster processing is
desired to create the packaged file more quickly.  It is not recommended for
production or general distribution of binary packages.

@noindent
A typical recipe contains the following variables:

@itemize @bullet
@item @samp{program}: Software name.

It matches the source name.  It is also used to compose the name of the
package if @samp{$@{pkgname@}} is not specified.

@item @samp{version}: Software version.

It matches the source name.  It is also used to compose the version of the
package if @samp{$@{pkgversion@}} is not specified.

@item @samp{arch}: Software architecture.

It is used to compose the architecture of the package in which it is
build.

@item @samp{release}: Release number.

This is used to reflect the release number of the package.  It is
recommended to increase this number after any significant change in
the recipe or post-install script.

@item @samp{pkgcategory}: Package category.

Optional but recommended variable to categorize the package name when it is
created.
@end itemize

@noindent
Obtaining sources over the network must be declared in the recipe using
the @samp{fetch} variable.

The variables @samp{netget} and @samp{rsync} can be defined in @file{qirc}
to establish a network downloader in order to get the sources.  If they
are not defined, qi uses default values:

@samp{netget} is the general network downloader tool, defaults sets to
@samp{wget2 -c -w1 -t3 --no-check-certificate}.

@samp{rsync} is the network tool for sources containing the prefix for
the RSYNC protocol, default sets to
@samp{rsync -v -a -L -z -i --progress}.

The variable @samp{description} is used to print the package description
when a package is installed.

A description has two parts: a brief description, and a long description.
By convention, the syntax of @samp{description} is:

@example
description="
Brief description.

Long description.
"
@end example

The first line of the value represented is a brief description of the
software (called "blurb").  A blank line separates the @emph{brief
description} from the @emph{long description}, which should contain a more
descriptive description of the software.

@noindent
An example looks like:

@example
description="
The GNU core utilities.

The GNU core utilities are the basic file, shell and text manipulation
utilities of the GNU operating system.  These are the core utilities
which are expected to exist on every operating system.
"
@end example

Please consider a length limit of 78 characters as maximum, because the same
one would be used on the meta file creation.  See
@ref{Recipes, The meta file} section.

The @samp{homepage} variable is used to declare the main site or home page:

@example
homepage=https://www.gnu.org/software/gcc
@end example

The variable @samp{license} is used for license information@footnote{
The proposal for @samp{license} was made by Richard M. Stallman at
@url{https://lists.gnu.org/archive/html/gnu-linux-libre/2016-05/msg00003.html}.}.
Some code in the program can be covered by license A, license B, or
license C.  For "separate licensing" or "heterogeneous licensing", we
suggest using @strong{|} for a disjunction, @strong{&} for a conjunction
(if that ever happens in a significant way), and comma for heterogeneous
licensing.  Comma would have lower precedence, plus added special terms.

@example
license="LGPL, GPL | Artistic - added permission"
@end example

@section Writing recipes
@cindex writing recipes

Originally, Qi was designed for the series of Dragora GNU/Linux-Libre 3;
this doesn't mean you can't use it in another distribution, just that if
you do, you'll have to try it out for yourself. To help with this, here
are some references to well-written recipes:

@itemize @bullet
@item @url{https://git.savannah.nongnu.org/cgit/dragora.git/tree/recipes}
@item @url{https://notabug.org/dragora/dragora/src/master/recipes}
@item @url{https://notabug.org/dragora/dragora-extras/src/master/recipes}
@item @url{https://git.savannah.nongnu.org/cgit/dragora/dragora-extras.git/tree/recipes}
@end itemize

@section Building packages
@cindex package build

A recipe is any valid regular file.  Qi sets priorities for reading a
recipe, the order in which qi looks for a recipe is:

@enumerate
@item
Current working directory.

@item
If the specified path name does not contain "recipe" as the last
component.  Qi will complete it by adding "recipe" to the path name.

@item
If the recipe is not in the current working directory, it will be
searched under @samp{$@{worktree@}/recipes}.  The last component will be
completed adding "recipe" to the specified path name.
@end enumerate

@noindent
To build a single package, type:

@example
qi build x-apps/xterm
@end example

@noindent
Multiple jobs can be passed to the compiler to speed up the build process:

@example
qi build --jobs 3 x-apps/xterm
@end example

@noindent
Update or install the produced package (if not already installed) when the
build command ends:

@example
qi build -j3 --upgrade x-apps/xterm
@end example

@noindent
Only process a recipe but do not create the binary package:

@example
qi build --no-package dict/aspell
@end example

The options --install or --upgrade have no effect when --no-package
is given.

@noindent
This is useful to inspect the build process of the above recipe:

qi build --keep --no-package dict/aspell 2>&1 | tee aspell-log.txt

The --keep option could preserve the source directory and the destination
directory for later inspection.  A log file of the build process will be
created redirecting both, standard error and standard output to tee(1).

@section Variables from the environment
@cindex environment variables

Qi has environment variables which can be used at build time:

The variable @env{TMPDIR} sets the temporary directory for sources, which is
used for package extractions (see @ref{Examining packages}) and is
prepended to the value of @samp{$@{srcdir@}} and @samp{$@{destdir@}} in
build command.  By convention its default value is equal to
@samp{/usr/src/qi/build}.

The variables @env{QICFLAGS}, @env{QICXXFLAGS}, @env{QILDFLAGS}, and
@env{QICPPFLAGS} have no effect by default.  The environment variables
such as @env{CFLAGS}, @env{CXXFLAGS}, @env{LDFLAGS}, and @env{CPPFLAGS}
are unset at compile time:

@noindent
Recommended practice is to set variables in the command line of
@samp{configure} or @emph{make(1)} instead of exporting to the
environment.  As follows:

@url{https://www.gnu.org/software/make/manual/html_node/Environment.html}
@quotation
It is not wise for makefiles to depend for their functioning on environment
variables set up outside their control, since this would cause different
users to get different results from the same makefile.  This is against the
whole purpose of most makefiles.
@end quotation

Setting environment variables for configure is deprecated because running
configure in varying environments can be dangerous.

@url{https://gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Defining-Variables.html}
@quotation
Variables not defined in a site shell script can be set in the environment
passed to configure.  However, some packages may run configure again
during the build, and the customized values of these variables may be
lost.  In order to avoid this problem, you should set them in the
configure command line, using @samp{VAR=value}.  For example:

@code{./configure CC=/usr/local2/bin/gcc}
@end quotation

@url{https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Setting-Output-Variables.html}
@quotation
If for instance the user runs @samp{CC=bizarre-cc ./configure}, then the cache,
config.h, and many other output files depend upon bizarre-cc being the C
compiler.  If for some reason the user runs ./configure again, or if it is
run via @samp{./config.status --recheck}, (See Automatic Remaking, and see
config.status Invocation), then the configuration can be inconsistent,
composed of results depending upon two different compilers.
[...]
Indeed, while configure can notice the definition of CC in @samp{./configure
CC=bizarre-cc}, it is impossible to notice it in @samp{CC=bizarre-cc
./configure}, which, unfortunately, is what most users do.
[...]
configure: error: changes in the environment can compromise the build.
@end quotation

If the @env{SOURCE_DATE_EPOCH} environment variable is set to a UNIX timestamp
(defined as the number of seconds, excluding leap seconds, since 01 Jan 1970
00:00:00 UTC.); then the given timestamp will be used to overwrite any newer
timestamps on the package contents (when it is created).  More information
about this can be found at
@uref{https://reproducible-builds.org/specs/source-date-epoch/}.

@section The meta file
@cindex the meta file

The "meta file" is a regular file created during the build process, it
contains information about the package such as package name, package
version, architecture, release, fetch address, description, and other
minor data extracted from processed recipes.  The name of the file is
generated as @samp{$@{full_pkgname@}.tlz.txt}, and its purpose is to
reflect essential information to the user without having to look inside
the package content.  The file format is also intended to be used by
other scripts or by common Unix tools.

The content of a meta file looks like:

@example
#
# Pattern scanning and processing language.
#
# The awk utility interprets a special-purpose programming language
# that makes it possible to handle simple data-reformatting jobs
# with just a few lines of code.  It is a free version of 'awk'.
#
# GNU awk implements the AWK utility which is part of
# IEEE Std 1003.1 Shell and Utilities (XCU).
#

QICFLAGS="-O2"
QICXXFLAGS="-O2"
QILDFLAGS=""
QICPPFLAGS=""
pkgname=gawk
pkgversion=5.0.1
arch=amd64
release=1
pkgcategory="tools"
full_pkgname=gawk_5.0.1_amd64-1@@tools
blurb="Pattern scanning and processing language."
homepage="https://www.gnu.org/software/gawk"
license="GPLv3+"
fetch="https://ftp.gnu.org/gnu/gawk/gawk-5.0.1.tar.lz"
replace=""
@end example

A package descriptions is extracted from the variable @samp{description}
where each line is interpreted literally and pre-formatted to fit in
(exactly) @strong{80 columns}, plus the character @samp{#} and a blank
space is prefixed to every line (shell comments).

@noindent
In addition to the Special variables, there are implicit variables such as
@samp{blurb}:

The @samp{blurb} variable is related to the special variable
@samp{description}.  Its value is made from the first (substantial)
line of @samp{description}, mentioned as the "brief description".

The build flags such as @samp{QICFLAGS}, @samp{QICXXFLAGS},
@samp{QILDFLAGS}, and @samp{QICPPFLAGS} are only added to the meta file
if the declared variable @samp{arch} is not equal to the "noarch" value.

@node Order files
@chapter Order files
@cindex handling build order

The order command has the purpose of resolving the build order through
.order files.  An order file contains a list of recipe names, by default
does not perform any action other than to print a resolved list in
descending order.  For example, if @strong{a} depends on @strong{b} and
@strong{c}, and @strong{c} depends on @strong{b} as well, the file might
look like:

@example
a: c b
b:
c: b
@end example

Each letter represents a recipe name, complete dependencies for
the first recipe name are listed in descending order, which is
printed from right to left, and removed from left to right:

@sc{Output}

@example
b
c
a
@end example

Blank lines, colons and parentheses are simply ignored.  Comment lines
beginning with @samp{#} are allowed.

@noindent
An order file could be used to build a series of packages, for example,
if the content is:

@example
# Image handling libraries

libs/libjpeg-turbo: devel/nasm
x-libs/jasper: libs/libjpeg-turbo
libs/tiff: libs/libjpeg-turbo
@end example

To proceed with each recipe, we can type:

@example
qi order imglibs.order | qi build --install -
@end example

The output of @samp{qi order imglibs.order} tells to qi in which order it
should build the recipes:

@example
devel/nasm
libs/libjpeg-turbo
x-libs/jasper
libs/tiff
@end example


@node Creating packages
@chapter Creating packages
@cindex package creation

The creation command is an internal function of qi to make new Qi
compatible packages.  A package is produced using the contents of
the Current Working Directory and the package file is written out.

@example
Usage: qi create [@var{Output/PackageName.tlz}]...
@end example

The argument for the file name to be written must contain a fully
qualified named directory as the output directory where the package
produced will be written.  The file name should be composed using the
full name: name-version-architecture-release[@@pkgcategory].tlz

@sc{Example}

@example
cd /usr/pkg
cd claws-mail_3.17.1_amd64-1@@x-apps
qi create /var/cache/qi/packages/claws-mail_3.17.1_amd64-1@@x-apps
@end example

In this case, the package "claws-mail_3.17.1_amd64-1@@x-apps" will be
written into @samp{/var/cache/qi/packages/}.

@noindent
All packages produced are complemented by a checksum file (.sha256).


@node Examining packages
@chapter Examining packages
@cindex package examination

The extraction command serves to examine binary packages for debugging
purposes. It decompresses a package into a single directory, verifying
its integrity and preserving all of its properties (owner and permissions).

@example
Usage: qi extract [@var{packagename.tlz}]...
@end example

@sc{Example}

@example
qi extract mksh_R56c_amd64-1@@shells.tlz
@end example

This action will put the content of "mksh_R56c_amd64-1@@shells.tlz" into a
single directory, this is a private directory for the user who requested
the action, creation operation will be equal to @strong{u=rwx,g=,o= (0700)}.
The package content will reside on this location, default mask to deploy
the content will be equal to @strong{u=rwx,g=rwx,o=rwx (0000)}.

@noindent
Note: the creation of the custom directory is influenced by the value
of the @env{TMPDIR} variable.


@node Qi exit status
@chapter Qi exit status
@cindex exit codes

All the exit codes are described in this chapter.

@table @samp
@item 0
Successful completion (no errors).

@item 1
Minor common errors:

@itemize @bullet
@item Help usage on invalid options or required arguments.

@item Program needed by qi (prerequisite) is not available.
@end itemize

@item 2
Command execution error:

This code is used to return the evaluation of an external command or shell
arguments in case of failure.

@item 3
Integrity check error for compressed files.

Compressed files means:

@itemize @bullet
@item A tarball file from tar(1), typically handled by the GNU tar implementation.
Supported extensions: .tar, .tar.gz, .tgz, .tar.Z, .tar.bz2, .tbz2, .tbz,
.tar.xz, .txz, .tar.zst, .tzst

@item A tarball file from tarlz(1).
Supported extensions: .tar.lz, .tlz

@item Zip files from unzip(1).
Supported extensions: .zip, .ZIP

@item Gzip files from gzip(1).
Supported extensions: .gz, .Z

@item Bzip2 files from bzip2(1).
Supported extension: .bz2

@item Lzip files from lzip(1).
Supported extension: .lz

@item Xz files from xz(1).
Supported extension: .xz

@item Zstd files from zstd(1).
Supported extension: .zst
@end itemize

@item 4
File empty, not regular, or expected.

It's commonly expected:

@itemize @bullet
@item An argument for giving commands.

@item A regular file or readable directory.

@item An expected extension: .tlz, .sha256, .order.

@item A protocol supported by the network downloader tool.
@end itemize

@item 5
Empty or not defined variable:

This code is used to report empty or undefined variables (usually
variables coming from a recipe or assigned arrays that are tested).

@item 6
Package already installed:

The package directory for an incoming .tlz package already exists.

@item 10
Network manager error:

This code is used if the network downloader tool fails for some reason.
@end table