tor-browser-build

Tor Browser Build
=================

Installing build dependencies
-----------------------------

To build Tor Browser, you need a recent Linux distribution with support
for user_namespaces(7) (such as Debian Buster, Ubuntu 16.04, Fedora 30,
etc ...). You will need to install the uidmap package, providing the
newuidmap and newgidmap commands.

The sources of most components are downloaded using git, which needs to
be installed. The mercurial package is also needed.

Zstandard (zstd) is used to compress some tarballs and needs to be
installed. You'll also need tar >= 1.31, for zstd support.

You also need a few perl modules installed:
- YAML::XS
- File::Basename
- Getopt::Long
- Template
- IO::Handle
- IO::CaptureOutput
- JSON
- File::Temp
- Path::Tiny
- File::Path
- File::Slurp
- File::Copy::Recursive
- String::ShellQuote
- Sort::Versions
- Digest::SHA
- Data::UUID
- Data::Dump
- DateTime
- XML::Writer
- Parallel::ForkManager

If you are running Debian or Ubuntu, you can install them with:

# apt-get install libdata-dump-perl libdata-uuid-perl libdatetime-perl \
libdigest-sha-perl libfile-copy-recursive-perl \
libfile-slurp-perl libio-all-perl libio-captureoutput-perl \
libio-handle-util-perl libjson-perl \
libparallel-forkmanager-perl libpath-tiny-perl \
libsort-versions-perl libstring-shellquote-perl \
libtemplate-perl libxml-libxml-perl libxml-writer-perl \
libyaml-libyaml-perl git mercurial uidmap zstd

If you are running an Arch based system, you should be able to install them with:

# pacman -S perl-datetime perl-path-tiny perl-yaml perl-yaml-libyaml \
perl-yaml-tiny perl-template-toolkit perl-io-captureoutput \
perl-file-copy-recursive perl-string-shellquote \
perl-sort-versions perl-data-uuid perl-data-dump perl-json \
perl-digest-sha1 perl-io-all perl-file-slurp perl-sys-syscall \
perl-parallel-forkmanager perl-xml-libxml perl-lwp-protocol-https \
zstd

On Arch based systems you also need to generate some .ph files RBM expects:

# cd /usr/include/sys; h2ph syscall.h
# cd /usr/include/; h2ph asm/unistd.h asm/unistd_64.h bits/syscall.h

The build system is based on rbm, which is included as a git submodule
in the rbm/ directory. You can fetch the rbm git submodule by running
'make submodule-update'.

The build uses user_namespaces(7), which are disabled by default on Debian.
To enable them you can use the following command as root:

# sysctl -w kernel.unprivileged_userns_clone=1

You can enable them permanently by adding the setting to /etc/sysctl.d/

The user you use to build needs to have a range of subordinate uids and
gids in /etc/subuid and /etc/subgid. Most of the time they are added by
default when the user is created. If it was not the case, you can use
usermod(8) with the --add-subuids and --add-subgids options. See also
the subuid(5) and subgid(5) man pages.


Starting a build
----------------

To start a build, run one of the following commands, depending on the
channel you want to build:

$ make torbrowser-release
$ make torbrowser-alpha
$ make torbrowser-nightly

You can find the build result in the directory
torbrowser/release/unsigned/$version or torbrowser/alpha/unsigned/$version
for release or alpha builds. The result of nightly can be found in the
torbrowser/nightly/$version directory.

If you want to build for a specific platform only, append the platform
name to the makefile target:

$ make torbrowser-nightly-linux-x86_64
$ make torbrowser-nightly-linux-i686
$ make torbrowser-nightly-windows-i686
$ make torbrowser-nightly-windows-x86_64
$ make torbrowser-nightly-macos
$ make torbrowser-nightly-android-armv7
$ make torbrowser-nightly-android-aarch64
$ make torbrowser-nightly-android-x86
$ make torbrowser-nightly-android-x86_64

When you want to quickly do a build to test a change, you can use the
testbuild makefile target, and find the build in the testbuild directory.
The build will be the same as regular alpha builds, except that in order
to make the build faster, only the en-US locale will be built, and no
mar file will be created. If you want to base your testbuild on the latest
nightly code insted, rename rbm.local.conf.example to rbm.local.conf
and adapt the torbrowser-testbuild option accordingly.

Similar makefile targets exist for building Base Browser and Privacy Browser
instead of Tor Browser. To build Base Browser, replace `torbrowser` by
`basebrowser` in the target name. For Privacy Browser, use `privacybrowser`.


Updating git sources
--------------------

You can run `make fetch` to fetch the latest sources from git for all
components included in Tor Browser. You should run this if you want to
make a nightly build with the latest commits, and you disabled automatic
fetching of new commits for nightly builds in rbm.local.conf.


Number of make processes
------------------------

By default `nproc` is used to determine the number of processes to run
simultaneously (with make -jN where N is the number returned by `nproc`).
If you want to change the number of processes used, you can set the
RBM_NUM_PROCS environment variable:

$ export RBM_NUM_PROCS=8

You can also set the num_procs option in rbm.local.conf.


Automated builds
----------------

If the build fails, a shell will automatically open in the build
container to help you debug the problem. You probably want to disable
this if you want to do automated builds. To disable this, set
the RBM_NO_DEBUG environment variable to 1:

export RBM_NO_DEBUG=1

Or set the debug option to 0 in the rbm.local.conf file.

If you want to select the output directory, you can use rbm's --output-dir
option. You can look at the Makefile to find the rbm command for what
you want to build, and add the --output-dir option. For example, if you
want to build Tor Browser nightly for linux-x86_64:

./rbm/rbm build release --output-dir=/var/builds/nightly/2020-05-23 \
--target nightly --target torbrowser-linux-x86_64

The files will be put in the directory selected by --output-dir in a
subdirectory named as the version number (or current date for nightly).
To remove this version subdirectory, add the noversiondir target:

./rbm/rbm build release --output-dir=/var/builds/nightly/2020-05-23 \
--target nightly --target torbrowser-linux-x86_64 \
--target noversiondir


Automated builds using tbb-testsuite
------------------------------------

The Tor Browser testsuite scripts can also be used to do nightly builds
and publish the build logs. This page has some information about the
setup we use for nightly builds:
https://gitlab.torproject.org/tpo/applications/tor-browser/-/wikis/Nightly_Builds_Setup


Signing builds
--------------

If the environment variable RBM_SIGN_BUILD is set to 1, the
sha256sums-unsigned-build.txt and sha256sums-unsigned-build.incrementals.txt
files will be signed with gpg. You can use the RBM_GPG_OPTS environment
variable to add some options to the gpg command used to sign the file.
You can also set the var/sign_build and var/sign_build_gpg_opts options
in the rbm.local.conf file.


Cleaning obsolete files and containers images
---------------------------------------------

You can run `make clean` to clean old build files and containers that
are no longer used in current builds. Before doing that, you need to
configure the branches and build targets you are using in the
rbm.local.conf file. The cleaning script will check out all the configured
branches to create a list of used build files, and delete the files
from the 'out' directory that are not used. If you want to see the list
of files and containers that would be removed without doing it, you can
use `make clean-dry-run`.


Building without containers (Android builds only)
-------------------------------------------------

By default the build is done inside containers. Adding the no_containers
target will disable the use of containers. The following commands can
be used to build the alpha version for e.g. android-armv7:

./rbm/rbm build release --target no_containers --target testbuild \
--target torbrowser-android-armv7

Note: the logs will still show the use and creation of a container image
called "containers_disabled". This is due to the way we disable the use
of containers: the container-image project is still called, but it will
just create an empty file instead of a real container image.

The build without containers is currently only supported for the Android
builds, and will require that you run Debian Buster and install build
dependencies for all the components that are built. This can be done
with the following command:

# apt-get install build-essential python automake libtool zip unzip \
autoconf2.13 openjdk-8-jdk gettext-base autotools-dev \
automake autoconf libtool autopoint libssl-dev \
pkg-config zlib1g-dev libparallel-forkmanager-perl \
libfile-slurp-perl bzip2 xz-utils apksigner yasm


Common Build Errors
-------------------

You can look at the file doc/BUILD_ERRORS.txt for a list of common build
errors and their solutions.


Hacking on the Tor Browser build
--------------------------------

The file doc/HACKING.txt tries to list the main things to know when
making changes to the Tor Browser build.


Description of makefile rules
-----------------------------

You can find a description of the Makefile rules in the file doc/MAKEFILE.txt.