qi-content.texi 38 KB

  1. @c -*-texinfo-*-
  2. @c qi-content.texi
  3. @c This is part of the Qi user guide.
  4. @c Copyright (C) 2019-2022 Matias Fonzo, <selk@dragora.org>.
  5. @c See the file qi-header.texi for copying conditions.
  6. @c
  7. @c If something is modified here, be aware of the increase of VERSION in
  8. @c qi-header.texi, this will produce the result of the manual: qi.info
  9. @node Introduction to Qi
  10. @chapter Introduction to Qi
  11. @cindex introduction to qi
  12. Qi is a simple but well-integrated package manager. It can create,
  13. install, remove, and upgrade software packages. Qi produces binary
  14. packages using recipes, which are files containing specific instructions
  15. to build each package from source. Qi can manage multiple packages
  16. under a single directory hierarchy. This method allows to maintain a set
  17. of packages and multiple versions of them. This means that Qi could be
  18. used as the main package manager or complement the existing one.
  19. Qi offers a friendly command line interface, a global configuration
  20. file, a simple recipe layout to deploy software packages; also works
  21. with binary packages in parallel, speeding up installations and packages
  22. in production. The format used for packages is a simplified and safer
  23. variant of POSIX pax archive compressed in lzip format.
  24. Qi is a modern (POSIX-compliant) shell script released under the
  25. terms of the GNU General Public License. There are only two major
  26. dependencies for the magic: graft(1) and tarlz(1), the rest is expected
  27. to be found in any Unix-like system.
  28. @node Invoking qi
  29. @chapter Invoking qi
  30. @cindex invocation
  31. This chapter describes the synopsis for invoking Qi.
  32. @example
  33. Usage: qi COMMAND [@var{OPTION}...] [@var{FILE}]...
  34. @end example
  35. @noindent
  36. One mandatory command specifies the operation that @samp{qi} should
  37. perform, options are meant to detail how this operation should be
  38. performed during or after the process.
  39. @noindent
  40. Qi supports the following commands:
  41. @table @code
  42. @item warn
  43. Warn about files that will be installed.
  44. @item install
  45. Install packages.
  46. @item remove
  47. Remove packages.
  48. @item upgrade
  49. Upgrade packages.
  50. @item extract
  51. Extract packages for debugging purposes.
  52. @item create
  53. Create a .tlz package from directory.
  54. @item build
  55. Build packages using recipe names.
  56. @item order
  57. Resolve build order through .order files
  58. @end table
  59. @noindent
  60. Options when installing, removing, or upgrading software packages:
  61. @table @code
  62. @item -f
  63. @itemx --force
  64. Force upgrade of pre-existing packages.
  65. @item -k
  66. @itemx --keep
  67. Keep directories when build/remove/upgrade.
  68. Keep (don't delete) the package directory when using remove/upgrade command.
  69. This will also try to preserve the directories @samp{$@{srcdir@}} and
  70. @samp{$@{destdir@}} when using build command. Its effect is available in
  71. recipes as @samp{$@{keep_srcdir@}} and @samp{$@{keep_destdir@}}. See
  72. @ref{Recipes, Special variables} for details.
  73. @item -p
  74. @itemx --prune
  75. Prune conflicts.
  76. @item -P
  77. @itemx --packagedir=<dir>
  78. Set directory for package installations.
  79. @item -t
  80. @itemx --targetdir=<dir>
  81. Set target directory for symbolic links.
  82. @item -r
  83. @itemx --rootdir=<dir>
  84. Use the fully qualified named directory as the root directory for all qi
  85. operations.
  86. Note: the target directory and the package directory will be
  87. relative to the specified directory, excepting the graft log file.
  88. @end table
  89. @noindent
  90. Options when building software packages using recipes:
  91. @table @code
  92. @item -a
  93. @itemx --architecture
  94. Set architecture name for the package.
  95. @item -j
  96. @itemx --jobs
  97. Parallel jobs for the compiler.
  98. This option sets the variable @samp{$@{jobs@}}. If not specified, default
  99. sets to 1.
  100. @item -S
  101. @itemx --skip-questions
  102. Skip questions on completed recipes.
  103. @item -1
  104. @itemx --increment
  105. Increment release number (@samp{$@{release@}} + 1).
  106. The effect of this option will be omitted if --no-package is being used.
  107. @item -n
  108. @itemx --no-package
  109. Do not create a .tlz package.
  110. @item -i
  111. @itemx --install
  112. Install package after the build.
  113. @item -u
  114. @itemx --upgrade
  115. Upgrade package after the build.
  116. @item -o
  117. @itemx --outdir=<dir>
  118. Where the packages produced will be written.
  119. This option sets the variable @samp{$@{outdir@}}.
  120. @item -w
  121. @itemx --worktree=<dir>
  122. Where archives, patches, recipes are expected.
  123. This option sets the variable @samp{$@{worktree@}}.
  124. @item -s
  125. @itemx --sourcedir=<dir>
  126. Where compressed sources will be found.
  127. This option sets the variable @samp{$@{tardir@}}.
  128. @end table
  129. @noindent
  130. Other options:
  131. @table @code
  132. @item -v
  133. @itemx --verbose
  134. Be verbose (an extra -v gives more).
  135. It sets the verbosity level, default sets to 0.
  136. The value 1 is used for more verbosity while the value 2 is too detailed.
  137. Although at the moment it is limited to graft(1) verbosity.
  138. @item -N
  139. @itemx --no-rc
  140. Do not read the configuration file.
  141. This will ignore reading the qirc file.
  142. @item -L
  143. @itemx --show-location
  144. Print default directory locations and exit.
  145. This will print the target directory, package directory, working tree,
  146. the directory for sources, and the output directory for the packages
  147. produced. The output will appear on STDOUT as follows:
  148. @example
  149. QI_TARGETDIR=/usr/local
  150. QI_PACKAGEDIR=/usr/local/pkgs
  151. QI_WORKTREE=/usr/src/qi
  152. QI_TARDIR=/usr/src/qi/sources
  153. QI_OUTDIR=/var/cache/qi/packages
  154. @end example
  155. You can set these environment variables using one of the following methods:
  156. @code{eval "$(qi -L)"}
  157. This will display the default locations taking into account the values set
  158. from the qirc configuration file. You can deny the influence of the
  159. configuration file by setting the option @samp{-N}.
  160. @code{eval "$(qi -N -L)"}
  161. Or you can adjust the new locations using the command-line options, e.g:
  162. @code{eval "$(qi -N --targetdir=/directory -L)"}
  163. @item -h
  164. @itemx --help
  165. Display the usage and exit.
  166. @item -V
  167. @itemx --version
  168. This will print the (short) version information and then exit.
  169. The same can be achieved if Qi is invoked as @samp{qi version}.
  170. @end table
  171. When FILE is -, qi can read from the standard input. See examples from
  172. the @ref{Packages} section.
  173. Exit status: 0 for a normal exit, 1 for minor common errors (help usage,
  174. support not available, etc), 2 to indicate a command execution error;
  175. 3 for integrity check error on compressed files, 4 for empty, not
  176. regular, or expected files, 5 for empty or not defined variables,
  177. 6 when a package already exist, 10 for network manager errors.
  178. For more details, see the @ref{Qi exit status} section.
  179. @node The qirc file
  180. @chapter The qirc file
  181. @cindex configuration file
  182. The global @file{qirc} file offers a way to define variables and tools
  183. (such as a download manager) for default use. This file is used by qi
  184. at runtime, e.g., to build, install, remove or upgrade packages.
  185. Variables and their possible values must be declared as any other
  186. variable in the shell.
  187. @noindent
  188. The command line options related to the package directory and target
  189. directory and some of the command line options used for the build command,
  190. have the power to override the values declared on @file{qirc}.
  191. See @ref{Invoking qi}.
  192. @noindent
  193. The order in which qi looks for this file is:
  194. @enumerate
  195. @item
  196. @env{$@{HOME@}/.qirc}
  197. @- Effective user.
  198. @item
  199. @samp{$@{sysconfdir@}/qirc}
  200. @- System-wide.
  201. @end enumerate
  202. If you intend to run qi as effective user, the file
  203. @samp{$@{sysconfdir@}/qirc} could be copied to @env{$@{HOME@}/.qirc}
  204. setting the paths for @samp{$@{packagedir@}} and @samp{$@{targetdir@}}
  205. according to the @env{$HOME}.
  206. @node Packages
  207. @chapter Packages
  208. @cindex managing packages
  209. A package is a suite of programs usually distributed in binary form
  210. which may also contain manual pages, documentation, or any other file
  211. associated to a specific software.
  212. The package format used by qi is a simplified POSIX pax archive
  213. compressed using lzip@footnote{For more details about tarlz and the
  214. lzip format, visit @url{https://lzip.nongnu.org/tarlz.html}.}. The
  215. file extension for packages ends in @samp{.tlz}.
  216. @noindent
  217. Both package installation and package de-installation are managed using
  218. two important (internal) variables: @samp{$@{packagedir@}} and
  219. @samp{$@{targetdir@}}, these values can be changed in the
  220. configuration file or via options.
  221. @samp{$@{packagedir@}} is a common directory tree where the package
  222. contents will be decompressed (will reside).
  223. @samp{$@{targetdir@}} is a target directory where the links will be
  224. made by graft(1) taking @samp{$@{packagedir@}/package_name} into account.
  225. @noindent
  226. Packages are installed in self-contained directory trees and symbolic
  227. links from a common area are made to the package files. This allows
  228. multiple versions of the same package to coexist on the same system.
  229. @section Package conflicts
  230. @cindex package conflicts
  231. All the links to install or remove a package are handled by graft(1).
  232. Since multiple packages can be installed or removed at the same time,
  233. certain conflicts may arise between the packages.
  234. @noindent
  235. graft@footnote{The official guide for Graft can be found at
  236. @url{https://peters.gormand.com.au/Home/tools/graft/graft.html}.}
  237. defines a CONFLICT as one of the following conditions:
  238. @itemize @bullet
  239. @item
  240. If the package object is a directory and the target object exists but is
  241. not a directory.
  242. @item
  243. If the package object is not a directory and the target object exists
  244. and is not a symbolic link.
  245. @item
  246. If the package object is not a directory and the target object exists
  247. and is a symbolic link to something other than the package object.
  248. @end itemize
  249. @noindent
  250. The default behavior of qi for an incoming package is to ABORT if a
  251. conflict arises. When a package is going to be deleted, qi tells to
  252. graft(1) to remove those parts that are not in conflict, leaving the
  253. links to the belonging package. This behavior can be forced if the
  254. --prune option is given.
  255. @section Installing packages
  256. @cindex package installation
  257. To install a single package, simply type:
  258. @example
  259. qi install coreutils_8.30_i586-1@@tools.tlz
  260. @end example
  261. @noindent
  262. To install multiple packages at once, type:
  263. @example
  264. qi install gcc_8.3.0_i586-1@@devel.tlz rafaela_2.2_i586-1@@legacy.tlz ...
  265. @end example
  266. @noindent
  267. Warn about the files that will be linked:
  268. @example
  269. qi warn bash_5.0_i586-1@@shells.tlz
  270. @end example
  271. This is to verify the content of a package before installing it.
  272. @noindent
  273. See the process of an installation:
  274. @example
  275. qi install --verbose mariana_3.0_i586-1@@woman.tlz
  276. @end example
  277. A second --verbose or -v option gives more (very verbose).
  278. @noindent
  279. Installing package in a different location:
  280. @example
  281. qi install --rootdir=/media/floppy lzip_1.21_i586-1@@compressors.tlz
  282. @end example
  283. Important: the --rootdir option assumes @samp{$@{targetdir@}} and
  284. @samp{$@{packagedir@}}. See the following example:
  285. @example
  286. qi install --rootdir=/home/selk lzip_1.21_i586-1@@compressors.tlz
  287. @end example
  288. The content of "lzip_1.21_i586-1@@compressors.tlz" will be decompressed
  289. into @samp{/home/selk/pkgs/lzip_1.21_i586-1@@compressors}.
  290. Assuming that the main binary for lzip is under
  291. @samp{/home/selk/pkgs/lzip_1.21_i586-1@@compressors/usr/bin/}
  292. the target for "usr/bin" will be created at @samp{/home/selk}. Considering
  293. that you have exported the @env{PATH} as @samp{$@{HOME@}/usr/bin}, now the
  294. system is able to see the recent lzip command.
  295. @noindent
  296. Installing from a list of packages using standard input:
  297. @example
  298. qi install - < PACKAGELIST.txt
  299. @end example
  300. Or in combination with another tool:
  301. @example
  302. sort -u PACKAGELIST.txt | qi install -
  303. @end example
  304. The sort command will read and sorts the list of declared packages,
  305. while trying to have unique entries for each statement. The output
  306. produced is captured by Qi to install each package.
  307. An example of a list containing package names is:
  308. @example
  309. /var/cache/qi/packages/amd64/tcl_8.6.9_amd64-1@@devel.tlz
  310. /var/cache/qi/packages/amd64/tk_8.6.9.1_amd64-1@@devel.tlz
  311. /var/cache/qi/packages/amd64/vala_0.42.3_amd64-1@@devel.tlz
  312. @end example
  313. @section Removing packages
  314. @cindex package de-installation
  315. To remove a package, simply type:
  316. @example
  317. qi remove xz_5.2.4_i586-1@@compressors.tlz
  318. @end example
  319. @noindent
  320. Remove command will match the package name using @samp{$@{packagedir@}} as
  321. prefix. For example, if the value of @samp{$@{packagedir@}} has been
  322. set to /usr/pkg, this will be equal to:
  323. @example
  324. qi remove /usr/pkg/xz_5.2.4_i586-1@@compressors
  325. @end example
  326. @noindent
  327. Detailed output:
  328. @example
  329. qi remove --verbose /usr/pkg/xz_5.2.4_i586-1@@compressors
  330. @end example
  331. A second --verbose or -v option gives more (very verbose).
  332. @noindent
  333. By default the remove command does not preserve a package directory after
  334. removing its links from @samp{$@{targetdir@}}, but this behavior can be
  335. changed if the --keep option is passed:
  336. @example
  337. qi remove --keep /usr/pkg/lzip_1.21_i586-1@@compressors
  338. @end example
  339. This means that the links to the package can be reactivated, later:
  340. @example
  341. cd /usr/pkg && graft -i lzip_1.21_i586-1@@compressors
  342. @end example
  343. @noindent
  344. Removing package from a different location:
  345. @example
  346. qi remove --rootdir=/home/cthulhu xz_5.2.4_i586-1@@compressors
  347. @end example
  348. @noindent
  349. Removing a package using standard input:
  350. @example
  351. echo vala_0.42.3_amd64-1@@devel | qi remove -
  352. @end example
  353. This will match with the package directory.
  354. @section Upgrading packages
  355. @cindex package upgrade
  356. The upgrade command inherits the properties of the installation and removal
  357. process. To make sure that a package is updated, the package is installed
  358. in a temporary directory taking @samp{$@{packagedir@}} into account. Once
  359. the incoming package is pre-installed, qi can proceed to search and delete
  360. packages that have the same name (considered as previous ones). Finally,
  361. the package is re-installed at its final location and the temporary
  362. directory is removed.
  363. Since updating a package can be crucial and so to perform a successful
  364. upgrade, from start to finish, you will want to ignore some important
  365. system signals during the upgrade process, those signals are SIGHUP,
  367. @noindent
  368. To upgrade a package, just type:
  369. @example
  370. qi upgrade gcc_9.0.1_i586-1@@devel.tlz
  371. @end example
  372. This will proceed to upgrade "gcc_9.0.1_i586-1@@devel" removing any other
  373. version of "gcc" (if any).
  374. @noindent
  375. If you want to keep the package directories of versions found during the
  376. upgrade process, just pass:
  377. @example
  378. qi upgrade --keep gcc_9.0.1_i586-1@@devel.tlz
  379. @end example
  380. @noindent
  381. To see the upgrade process:
  382. @example
  383. qi upgrade --verbose gcc_9.0.1_i586-1@@devel.tlz
  384. @end example
  385. A second --verbose or -v option gives more (very verbose).
  386. @noindent
  387. To force the upgrade of an existing package:
  388. @example
  389. qi upgrade --force gcc_9.0.1_i586-1@@devel.tlz
  390. @end example
  391. @subsection Package blacklist
  392. @cindex package blacklist
  393. To implement general package facilities, either to install, remove or
  394. maintain the hierarchy of packages in a clean manner, qi makes use of the
  395. pruning operation via graft(1) by default:
  396. There is a risk if those are crucial packages for the proper functioning
  397. of the system, because it implies the deactivation of symbolic from the
  398. target directory, @emph{especially} when transitioning an incoming package
  399. into its final location during an upgrade.
  400. @noindent
  401. A blacklist of package names has been devised for the case where
  402. a user decides to upgrade all the packages in the system, or
  403. just the crucial ones, such as the C library.
  404. The blacklist is related to the upgrade command only, consists in installing
  405. a package instead of updating it or removing previous versions of it;
  406. the content of the package will be updated over the existing content at
  407. @samp{$@{packagedir@}}, while the existing links from
  408. @samp{$@{targetdir@}} will be preserved. A pruning of links will be
  409. carried out in order to re-link possible differences with the recent
  410. content, this helps to avoid leaving dead links in the target directory.
  411. @noindent
  412. Package names for the blacklist to be declared must be set from the
  413. configuration file. By default, it is declared using the package name,
  414. which is more than enough for critical system packages, but if you want to
  415. be more specific, you can declare a package using:
  416. @samp{$@{pkgname@}_$@{pkgversion@}_$@{arch@}-$@{release@}} where
  417. the package category is avoided for common matching. See
  418. @ref{Recipes, Special variables} for a description of these variables.
  419. @node Recipes
  420. @chapter Recipes
  421. @cindex recipes
  422. A recipe is a file telling qi what to do. Most often, the recipe tells
  423. qi how to build a binary package from a source tarball.
  424. A recipe has two parts: a list of variable definitions and a list of
  425. sections. By convention, the syntax of a section is:
  426. @example
  427. section_name()
  428. @{
  429. section lines
  430. @}
  431. @end example
  432. The section name is followed by parentheses, one newline and an opening
  433. brace. The line finishing the section contains just a closing brace.
  434. The section names or the function names currently recognized are
  435. @samp{build}.
  436. The @samp{build} section (or @strong{shell function}) is an augmented
  437. shell script that contains the main instructions to build software
  438. from source.
  439. If there are other functions defined by the packager, Qi detects them
  440. for later execution.
  441. @section Variables
  442. @cindex variables
  443. A "variable" is a @strong{shell variable} defined either in @file{qirc}
  444. or in a recipe to represent a string of text, called the variable's
  445. "value". These values are substituted by explicit request in the
  446. definitions of other variables or in calls to external commands.
  447. Variables can represent lists of file names, options to pass to
  448. compilers, programs to run, directories to look in for source files,
  449. directories to write output to, or anything else you can imagine.
  450. Definitions of variables in qi have four levels of precedence.
  451. Options which define variables from the command-line override those
  452. specified in the @file{qirc} file, while variables defined in the recipe
  453. override those specified in @file{qirc}, taking priority over those
  454. variables set by command-line options. Finally, the variables have
  455. default values if they are not defined anywhere.
  456. Options that set variables through the command-line can only reference
  457. variables defined in @file{qirc} and variables with default values.
  458. Definitions of variables in @file{qirc} can only reference variables
  459. previously defined in @file{qirc} and variables with default values.
  460. Definitions of variables in the recipe can only reference variables
  461. set by the command-line, variables previously defined in the recipe,
  462. variables defined in @file{qirc}, and variables with default values.
  463. @section Special variables
  464. @cindex special variables
  465. There are variables which can only be set using the command line options or
  466. via @file{qirc}, there are other special variables which can be defined or
  467. redefined in a recipe. See the following definitions:
  468. @samp{outdir} is the directory where the packages produced are written.
  469. This variable can be redefined per-recipe. Default sets to
  470. @samp{/var/cache/qi/packages}.
  471. @samp{worktree} is the working tree where archives, patches, and recipes
  472. are expected. This variable can not be redefined in the recipe. Default
  473. sets to @samp{/usr/src/qi}.
  474. @samp{tardir} is defined in the recipe to the directory where the tarball
  475. containing the source can be found. The full name of the tarball is
  476. composed as @samp{$@{tardir@}/$tarname}. Its value is available in the
  477. recipe as @samp{$@{tardir@}}; a value of . for @samp{tardir} sets it to
  478. the value of CWD (Current Working Directory), this is where the recipe
  479. lives.
  480. @samp{arch} is the architecture to compose the package name. Its value is
  481. available in the recipe as @samp{$@{arch@}}. Default value is the one
  482. that was set in the Qi configuration.
  483. @samp{jobs} is the number of parallel jobs to pass to the compiler. Its
  484. value is available in the recipe as @samp{$@{jobs@}}. The default value
  485. is 1.
  486. The two variables @samp{$@{srcdir@}} and @samp{$@{destdir@}} can be
  487. set in the recipe, as any other variable, but if they are not, qi uses
  488. default values for them when building a package.
  489. @samp{srcdir} contains the source code to be compiled, and defaults to
  490. @samp{$@{program@}-$@{version@}}. @samp{destdir} is the place where the
  491. built package will be installed, and defaults to
  492. @samp{$@{TMPDIR@}/package-$@{program@}}.
  493. If @samp{pkgname} is left undefined, the special variable @samp{program}
  494. is assigned by default. If @samp{pkgversion} is left undefined, the
  495. special variable @samp{version} is assigned by default.
  496. @samp{pkgname} and @samp{pkgversion} along with: @samp{version}, @samp{arch},
  497. @samp{release}, and (optionally) @samp{pkgcategory} are used to produce the
  498. package name in the form:
  499. @samp{$@{pkgname@}_$@{pkgversion@}_$@{arch@}-$@{release@}[@@$@{pkgcategory@}].tlz}
  500. @samp{pkgcategory} is an optional special variable that can be defined on the
  501. recipe to categorize the package name. If it is defined, then the
  502. package output will be composed as
  503. @samp{$@{pkgname@}_$@{pkgversion@}_$@{arch@}-$@{release@}[@@$@{pkgcategory@}.tlz}.
  504. Automatically, the value of @samp{pkgcategory} will be prefixed using the
  505. @samp{@@} (at) symbol which will be added to the last part of the package name.
  506. A special variable called @samp{replace} can be used to declare package names
  507. that will be replaced at installation time.
  508. The special variables @samp{keep_srcdir} and @samp{keep_destdir} are provided
  509. in order to preserve the directories @samp{$@{srcdir@}} or @samp{$@{destdir@}},
  510. if those exists as such. Note: The declaration of these variables are subject
  511. to manual deactivation; its purpose in recipes is to preserve the directories
  512. that relate to the package's build (source) and destination directory, that is
  513. so that another recipe can get a new package (or meta package) from there. For
  514. example, the declarations can be done as:
  515. @example
  516. keep_srcdir=keep_srcdir
  517. keep_destdir=keep_destdir
  518. @end example
  519. Then from another recipe you would proceed to copy the necessary files that
  520. will compose the meta package, from the main function you must deactivate
  521. the variables at the end:
  522. @example
  523. unset -v keep_srcdir keep_destdir
  524. @end example
  525. This will leave the 'keep_srcdir' and 'keep_destdir' variables blank to
  526. continue with the rest of the recipes.
  527. The special variable @samp{opt_skiprecipe} is available when you need to
  528. ignore a recipe cleanly, continuing with the next recipe. May you add a
  529. conditional test then set it as @samp{opt_skiprecipe=opt_skiprecipe}.
  530. The variable @samp{tarlz_compression_options} can be used to change the
  531. default compression options in tarlz(1), default sets to @samp{-9 --solid}.
  532. For example if the variable is declared as:
  533. @example
  534. tarlz_compression_options="-0 --bsolid"
  535. @end example
  536. It will change the granularity of tarlz(1) by using the @samp{--bsolid}
  537. option @footnote{About the @samp{--bsolid} granularity option of tarlz(1),
  538. @url{https://www.nongnu.org/lzip/manual/tarlz_manual.html#g_t_002d_002dbsolid}.},
  539. as well as increasing the compression speed by lowering the compression
  540. level with @samp{-0}.
  541. This is only recommended for recipes where testing, or faster processing is
  542. desired to create the packaged file more quickly. It is not recommended for
  543. production or general distribution of binary packages.
  544. @noindent
  545. A typical recipe contains the following variables:
  546. @itemize @bullet
  547. @item @samp{program}: Software name.
  548. It matches the source name. It is also used to compose the name of the
  549. package if @samp{$@{pkgname@}} is not specified.
  550. @item @samp{version}: Software version.
  551. It matches the source name. It is also used to compose the version of the
  552. package if @samp{$@{pkgversion@}} is not specified.
  553. @item @samp{arch}: Software architecture.
  554. It is used to compose the architecture of the package in which it is
  555. build.
  556. @item @samp{release}: Release number.
  557. This is used to reflect the release number of the package. It is
  558. recommended to increase this number after any significant change in
  559. the recipe or post-install script.
  560. @item @samp{pkgcategory}: Package category.
  561. Optional but recommended variable to categorize the package name when it is
  562. created.
  563. @end itemize
  564. @noindent
  565. Obtaining sources over the network must be declared in the recipe using
  566. the @samp{fetch} variable.
  567. The variables @samp{netget} and @samp{rsync} can be defined in @file{qirc}
  568. to establish a network downloader in order to get the sources. If they
  569. are not defined, qi uses default values:
  570. @samp{netget} is the general network downloader tool, defaults sets to
  571. @samp{wget2 -c -w1 -t3 --no-check-certificate}.
  572. @samp{rsync} is the network tool for sources containing the prefix for
  573. the RSYNC protocol, default sets to
  574. @samp{rsync -v -a -L -z -i --progress}.
  575. The variable @samp{description} is used to print the package description
  576. when a package is installed.
  577. A description has two parts: a brief description, and a long description.
  578. By convention, the syntax of @samp{description} is:
  579. @example
  580. description="
  581. Brief description.
  582. Long description.
  583. "
  584. @end example
  585. The first line of the value represented is a brief description of the
  586. software (called "blurb"). A blank line separates the @emph{brief
  587. description} from the @emph{long description}, which should contain a more
  588. descriptive description of the software.
  589. @noindent
  590. An example looks like:
  591. @example
  592. description="
  593. The GNU core utilities.
  594. The GNU core utilities are the basic file, shell and text manipulation
  595. utilities of the GNU operating system. These are the core utilities
  596. which are expected to exist on every operating system.
  597. "
  598. @end example
  599. Please consider a length limit of 78 characters as maximum, because the same
  600. one would be used on the meta file creation. See
  601. @ref{Recipes, The meta file} section.
  602. The @samp{homepage} variable is used to declare the main site or home page:
  603. @example
  604. homepage=https://www.gnu.org/software/gcc
  605. @end example
  606. The variable @samp{license} is used for license information@footnote{
  607. The proposal for @samp{license} was made by Richard M. Stallman at
  608. @url{https://lists.gnu.org/archive/html/gnu-linux-libre/2016-05/msg00003.html}.}.
  609. Some code in the program can be covered by license A, license B, or
  610. license C. For "separate licensing" or "heterogeneous licensing", we
  611. suggest using @strong{|} for a disjunction, @strong{&} for a conjunction
  612. (if that ever happens in a significant way), and comma for heterogeneous
  613. licensing. Comma would have lower precedence, plus added special terms.
  614. @example
  615. license="LGPL, GPL | Artistic - added permission"
  616. @end example
  617. @section Writing recipes
  618. @cindex writing recipes
  619. Originally, Qi was designed for the series of Dragora GNU/Linux-Libre 3;
  620. this doesn't mean you can't use it in another distribution, just that if
  621. you do, you'll have to try it out for yourself. To help with this, here
  622. are some references to well-written recipes:
  623. @itemize @bullet
  624. @item @url{https://git.savannah.nongnu.org/cgit/dragora.git/tree/recipes}
  625. @item @url{https://notabug.org/dragora/dragora/src/master/recipes}
  626. @item @url{https://notabug.org/dragora/dragora-extras/src/master/recipes}
  627. @item @url{https://git.savannah.nongnu.org/cgit/dragora/dragora-extras.git/tree/recipes}
  628. @end itemize
  629. @section Building packages
  630. @cindex package build
  631. A recipe is any valid regular file. Qi sets priorities for reading a
  632. recipe, the order in which qi looks for a recipe is:
  633. @enumerate
  634. @item
  635. Current working directory.
  636. @item
  637. If the specified path name does not contain "recipe" as the last
  638. component. Qi will complete it by adding "recipe" to the path name.
  639. @item
  640. If the recipe is not in the current working directory, it will be
  641. searched under @samp{$@{worktree@}/recipes}. The last component will be
  642. completed adding "recipe" to the specified path name.
  643. @end enumerate
  644. @noindent
  645. To build a single package, type:
  646. @example
  647. qi build x-apps/xterm
  648. @end example
  649. @noindent
  650. Multiple jobs can be passed to the compiler to speed up the build process:
  651. @example
  652. qi build --jobs 3 x-apps/xterm
  653. @end example
  654. @noindent
  655. Update or install the produced package (if not already installed) when the
  656. build command ends:
  657. @example
  658. qi build -j3 --upgrade x-apps/xterm
  659. @end example
  660. @noindent
  661. Only process a recipe but do not create the binary package:
  662. @example
  663. qi build --no-package dict/aspell
  664. @end example
  665. The options --install or --upgrade have no effect when --no-package
  666. is given.
  667. @noindent
  668. This is useful to inspect the build process of the above recipe:
  669. qi build --keep --no-package dict/aspell 2>&1 | tee aspell-log.txt
  670. The --keep option could preserve the source directory and the destination
  671. directory for later inspection. A log file of the build process will be
  672. created redirecting both, standard error and standard output to tee(1).
  673. @section Variables from the environment
  674. @cindex environment variables
  675. Qi has environment variables which can be used at build time:
  676. The variable @env{TMPDIR} sets the temporary directory for sources, which is
  677. used for package extractions (see @ref{Examining packages}) and is
  678. prepended to the value of @samp{$@{srcdir@}} and @samp{$@{destdir@}} in
  679. build command. By convention its default value is equal to
  680. @samp{/usr/src/qi/build}.
  681. The variables @env{QICFLAGS}, @env{QICXXFLAGS}, @env{QILDFLAGS}, and
  682. @env{QICPPFLAGS} have no effect by default. The environment variables
  683. such as @env{CFLAGS}, @env{CXXFLAGS}, @env{LDFLAGS}, and @env{CPPFLAGS}
  684. are unset at compile time:
  685. @noindent
  686. Recommended practice is to set variables in the command line of
  687. @samp{configure} or @emph{make(1)} instead of exporting to the
  688. environment. As follows:
  689. @url{https://www.gnu.org/software/make/manual/html_node/Environment.html}
  690. @quotation
  691. It is not wise for makefiles to depend for their functioning on environment
  692. variables set up outside their control, since this would cause different
  693. users to get different results from the same makefile. This is against the
  694. whole purpose of most makefiles.
  695. @end quotation
  696. Setting environment variables for configure is deprecated because running
  697. configure in varying environments can be dangerous.
  698. @url{https://gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Defining-Variables.html}
  699. @quotation
  700. Variables not defined in a site shell script can be set in the environment
  701. passed to configure. However, some packages may run configure again
  702. during the build, and the customized values of these variables may be
  703. lost. In order to avoid this problem, you should set them in the
  704. configure command line, using @samp{VAR=value}. For example:
  705. @code{./configure CC=/usr/local2/bin/gcc}
  706. @end quotation
  707. @url{https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Setting-Output-Variables.html}
  708. @quotation
  709. If for instance the user runs @samp{CC=bizarre-cc ./configure}, then the cache,
  710. config.h, and many other output files depend upon bizarre-cc being the C
  711. compiler. If for some reason the user runs ./configure again, or if it is
  712. run via @samp{./config.status --recheck}, (See Automatic Remaking, and see
  713. config.status Invocation), then the configuration can be inconsistent,
  714. composed of results depending upon two different compilers.
  715. [...]
  716. Indeed, while configure can notice the definition of CC in @samp{./configure
  717. CC=bizarre-cc}, it is impossible to notice it in @samp{CC=bizarre-cc
  718. ./configure}, which, unfortunately, is what most users do.
  719. [...]
  720. configure: error: changes in the environment can compromise the build.
  721. @end quotation
  722. If the @env{SOURCE_DATE_EPOCH} environment variable is set to a UNIX timestamp
  723. (defined as the number of seconds, excluding leap seconds, since 01 Jan 1970
  724. 00:00:00 UTC.); then the given timestamp will be used to overwrite any newer
  725. timestamps on the package contents (when it is created). More information
  726. about this can be found at
  727. @uref{https://reproducible-builds.org/specs/source-date-epoch/}.
  728. @section The meta file
  729. @cindex the meta file
  730. The "meta file" is a regular file created during the build process, it
  731. contains information about the package such as package name, package
  732. version, architecture, release, fetch address, description, and other
  733. minor data extracted from processed recipes. The name of the file is
  734. generated as @samp{$@{full_pkgname@}.tlz.txt}, and its purpose is to
  735. reflect essential information to the user without having to look inside
  736. the package content. The file format is also intended to be used by
  737. other scripts or by common Unix tools.
  738. The content of a meta file looks like:
  739. @example
  740. #
  741. # Pattern scanning and processing language.
  742. #
  743. # The awk utility interprets a special-purpose programming language
  744. # that makes it possible to handle simple data-reformatting jobs
  745. # with just a few lines of code. It is a free version of 'awk'.
  746. #
  747. # GNU awk implements the AWK utility which is part of
  748. # IEEE Std 1003.1 Shell and Utilities (XCU).
  749. #
  750. QICFLAGS="-O2"
  751. QICXXFLAGS="-O2"
  752. QILDFLAGS=""
  753. QICPPFLAGS=""
  754. pkgname=gawk
  755. pkgversion=5.0.1
  756. arch=amd64
  757. release=1
  758. pkgcategory="tools"
  759. full_pkgname=gawk_5.0.1_amd64-1@@tools
  760. blurb="Pattern scanning and processing language."
  761. homepage="https://www.gnu.org/software/gawk"
  762. license="GPLv3+"
  763. fetch="https://ftp.gnu.org/gnu/gawk/gawk-5.0.1.tar.lz"
  764. replace=""
  765. @end example
  766. A package descriptions is extracted from the variable @samp{description}
  767. where each line is interpreted literally and pre-formatted to fit in
  768. (exactly) @strong{80 columns}, plus the character @samp{#} and a blank
  769. space is prefixed to every line (shell comments).
  770. @noindent
  771. In addition to the Special variables, there are implicit variables such as
  772. @samp{blurb}:
  773. The @samp{blurb} variable is related to the special variable
  774. @samp{description}. Its value is made from the first (substantial)
  775. line of @samp{description}, mentioned as the "brief description".
  776. The build flags such as @samp{QICFLAGS}, @samp{QICXXFLAGS},
  777. @samp{QILDFLAGS}, and @samp{QICPPFLAGS} are only added to the meta file
  778. if the declared variable @samp{arch} is not equal to the "noarch" value.
  779. @node Order files
  780. @chapter Order files
  781. @cindex handling build order
  782. The order command has the purpose of resolving the build order through
  783. .order files. An order file contains a list of recipe names, by default
  784. does not perform any action other than to print a resolved list in
  785. descending order. For example, if @strong{a} depends on @strong{b} and
  786. @strong{c}, and @strong{c} depends on @strong{b} as well, the file might
  787. look like:
  788. @example
  789. a: c b
  790. b:
  791. c: b
  792. @end example
  793. Each letter represents a recipe name, complete dependencies for
  794. the first recipe name are listed in descending order, which is
  795. printed from right to left, and removed from left to right:
  796. @sc{Output}
  797. @example
  798. b
  799. c
  800. a
  801. @end example
  802. Blank lines, colons and parentheses are simply ignored. Comment lines
  803. beginning with @samp{#} are allowed.
  804. @noindent
  805. An order file could be used to build a series of packages, for example,
  806. if the content is:
  807. @example
  808. # Image handling libraries
  809. libs/libjpeg-turbo: devel/nasm
  810. x-libs/jasper: libs/libjpeg-turbo
  811. libs/tiff: libs/libjpeg-turbo
  812. @end example
  813. To proceed with each recipe, we can type:
  814. @example
  815. qi order imglibs.order | qi build --install -
  816. @end example
  817. The output of @samp{qi order imglibs.order} tells to qi in which order it
  818. should build the recipes:
  819. @example
  820. devel/nasm
  821. libs/libjpeg-turbo
  822. x-libs/jasper
  823. libs/tiff
  824. @end example
  825. @node Creating packages
  826. @chapter Creating packages
  827. @cindex package creation
  828. The creation command is an internal function of qi to make new Qi
  829. compatible packages. A package is produced using the contents of
  830. the Current Working Directory and the package file is written out.
  831. @example
  832. Usage: qi create [@var{Output/PackageName.tlz}]...
  833. @end example
  834. The argument for the file name to be written must contain a fully
  835. qualified named directory as the output directory where the package
  836. produced will be written. The file name should be composed using the
  837. full name: name-version-architecture-release[@@pkgcategory].tlz
  838. @sc{Example}
  839. @example
  840. cd /usr/pkg
  841. cd claws-mail_3.17.1_amd64-1@@x-apps
  842. qi create /var/cache/qi/packages/claws-mail_3.17.1_amd64-1@@x-apps
  843. @end example
  844. In this case, the package "claws-mail_3.17.1_amd64-1@@x-apps" will be
  845. written into @samp{/var/cache/qi/packages/}.
  846. @noindent
  847. All packages produced are complemented by a checksum file (.sha256).
  848. @node Examining packages
  849. @chapter Examining packages
  850. @cindex package examination
  851. The extraction command serves to examine binary packages for debugging
  852. purposes. It decompresses a package into a single directory, verifying
  853. its integrity and preserving all of its properties (owner and permissions).
  854. @example
  855. Usage: qi extract [@var{packagename.tlz}]...
  856. @end example
  857. @sc{Example}
  858. @example
  859. qi extract mksh_R56c_amd64-1@@shells.tlz
  860. @end example
  861. This action will put the content of "mksh_R56c_amd64-1@@shells.tlz" into a
  862. single directory, this is a private directory for the user who requested
  863. the action, creation operation will be equal to @strong{u=rwx,g=,o= (0700)}.
  864. The package content will reside on this location, default mask to deploy
  865. the content will be equal to @strong{u=rwx,g=rwx,o=rwx (0000)}.
  866. @noindent
  867. Note: the creation of the custom directory is influenced by the value
  868. of the @env{TMPDIR} variable.
  869. @node Qi exit status
  870. @chapter Qi exit status
  871. @cindex exit codes
  872. All the exit codes are described in this chapter.
  873. @table @samp
  874. @item 0
  875. Successful completion (no errors).
  876. @item 1
  877. Minor common errors:
  878. @itemize @bullet
  879. @item Help usage on invalid options or required arguments.
  880. @item Program needed by qi (prerequisite) is not available.
  881. @end itemize
  882. @item 2
  883. Command execution error:
  884. This code is used to return the evaluation of an external command or shell
  885. arguments in case of failure.
  886. @item 3
  887. Integrity check error for compressed files.
  888. Compressed files means:
  889. @itemize @bullet
  890. @item A tarball file from tar(1), typically handled by the GNU tar implementation.
  891. Supported extensions: .tar, .tar.gz, .tgz, .tar.Z, .tar.bz2, .tbz2, .tbz,
  892. .tar.xz, .txz, .tar.zst, .tzst
  893. @item A tarball file from tarlz(1).
  894. Supported extensions: .tar.lz, .tlz
  895. @item Zip files from unzip(1).
  896. Supported extensions: .zip, .ZIP
  897. @item Gzip files from gzip(1).
  898. Supported extensions: .gz, .Z
  899. @item Bzip2 files from bzip2(1).
  900. Supported extension: .bz2
  901. @item Lzip files from lzip(1).
  902. Supported extension: .lz
  903. @item Xz files from xz(1).
  904. Supported extension: .xz
  905. @item Zstd files from zstd(1).
  906. Supported extension: .zst
  907. @end itemize
  908. @item 4
  909. File empty, not regular, or expected.
  910. It's commonly expected:
  911. @itemize @bullet
  912. @item An argument for giving commands.
  913. @item A regular file or readable directory.
  914. @item An expected extension: .tlz, .sha256, .order.
  915. @item A protocol supported by the network downloader tool.
  916. @end itemize
  917. @item 5
  918. Empty or not defined variable:
  919. This code is used to report empty or undefined variables (usually
  920. variables coming from a recipe or assigned arrays that are tested).
  921. @item 6
  922. Package already installed:
  923. The package directory for an incoming .tlz package already exists.
  924. @item 10
  925. Network manager error:
  926. This code is used if the network downloader tool fails for some reason.
  927. @end table