Rel4tion-Wiki/maint/admin/ikiwiki/Git_Server.mdwn

Web editing works now, but the only way to make changes via git is to push to
the local repository stored in the ikiwiki user's home directory. We are going
to decouple the git repository from ikiwiki's clone - regardless of whether they
are physically on the same machine or not. Moving them around later will be
easy, because the setup doesn't rely on them being on the same machine, and
doesn't require them to be remote.

By default, the bare master repository uses a post-update git hook, which causes
it to update ikiwiki's clone whenever someone pushes an update. Ikiwiki's clone
is configured to use the bare repo as its remote "origin", i.e. when a change is
made via the web interface, it is pushed to the bare repo.

In our new setup, the bare repo will not touch ikiwiki's clone: Instead, it will
send an HTTP ping, which will cause ikiwiki to pull updates from the bare repo
and regenerate HTML. And ikiwiki will access the bare repo using SSH, rather
than directly using a file-based URI. Settings domain names to "localhost"
allows this setup to work for a single machine setup, while remaining very
flexible.

I assume you already have a git server. If you don't, see the git server
tutorial. I will also assume both repsitories are on the same machine, so the
domain name used is 'localhost'. If you use separate remote machines, replace
'localhost' with your IP addresses or domain names. A special case is I2P:
You'll need to create tunnels for it, so use 'localhost' anyway. I will use my
tunnel ports here:

- `localhost:8950` is a client tunnel for SSH, pointing to the SSH server on the
  machine where the git server is
- `localhost:8951` is a client tunnel for the git protocol, pointing to the git
  daemon on the machine where the git server is

If you don't use I2P, use your domain name (e.g.
ssh://git@git.hello-world.net/wiki.git) for SSH (the port will be 22, but it's
the default so you don't need to specify it) and the same (e.g.
git://git.hello-world.net/wiki.git) for git daemon (the port will be 9418, but
it's the default so you don't need to specify it). The domain in this example is
"git.hello-world.net" but can also be "localhost" if ikiwiki and the git server
are on the same machine.

First, we will need to give the ikiwiki user an SSH key, with which it will pull
and push to the bare repository located on the git server:

	# su - ikiwiki
	$ ssh-keygen -t rsa

It will ask several questions. Use the default name for the key. Also it is
__important that you use a blank password__, i.e. just press Enter when asked
for a password. It is required because the key will be used from a script.

Now create a new bare repository on the git server. The git server tutorial
explains how to do it with Gitolite. Assume we call the new repository 'wiki'.
Give the admin user RW+ access to it. You can give R access to 'daemon' and
provide a description, so that it is accessible via git:// protocol and appears
on Gitweb. Commit the push the change to the git server.

The next step is to add the ikiwiki user. First, copy ikiwiki's public SSH key
(`/home/ikiwiki/.ssh/id_rsa.pub`) to the git server, i.e. add a new user with
key 'ikiwiki.pub'. The git server tutorial explains exactly how. In the same
commit we will also give it the access it needs: Give user 'ikiwiki' RW access
to the 'wiki' repo. Now commit and push.

Now we need to tell ikiwiki to use 'wiki' as its remote origin, instead of
MyWiki.git which ikiwiki generated as a default. As the ikiwiki user, open file
`/home/ikiwiki/MyWiki/.git/config`. It should look more or less like this:

	[core]
		repositoryformatversion = 0
		filemode = true
		bare = false
		logallrefupdates = true
	[remote "origin"]
		url = /home/ikiwiki/MyWiki.git
		fetch = +refs/heads/*:refs/remotes/origin/*
	[branch "master"]
		merge = refs/heads/master
		remote = origin

See the 'url' setting? We need to change it to point to the new repository we
created on the git server. In my case, with I2P as described earlier, the new
value is `ssh://git@localhost:8950/wiki.git`. 'git' is the name of the git user
on the git server, 'localhost' is used because I have a local client tunnel
pointing to the git server, 8950 is the port of the local tunnel and 'wiki.git'
is there because we called the new repository 'wiki'. Gitolite appends '.git' to
the names of repositories (this is a convention for bare repositories).

After changing the 'url' and saving the file, it's time to test the
configuration. We're not done yet, but let's make sure ikiwiki can pull and push
to the new repository.

As the ikiwiki user, `cd` to the srcdir repo - in our case
`/home/ikiwiki/MyWiki` - and try to `git push origin master`. You will probably
be asked about the authenticity of the SSH server - check the fingerprint if you
want ([[TODO|TODO/OPEN]] how?). Once you accept the SSH server, the push operation
should work. A new 'master' branch will be created on the 'wiki' repo.

	# su - ikiwiki
	$ cd MyWiki
	$ git push origin master

Now try pulling. It should say "everything is up to date". On your personal
computer - assuming you gave yourself RW (or RW+) access to 'wiki' - clone the
'wiki' repository, make some trivial change to the index page (index.mdwn should
already be in the repository because you made edits via the web interface
earlier) and push them. Now, as the ikiwiki user on the machine where ikiwiki is
located, try `git pull` to make sure you receive the change.

	$ git pull

Both pulling and pushing should work without user interaction - no password
requested, etc. They both will be used from a script, so it's important.

As the ikiwiki user, open the setup file - in our case
`/home/ikiwiki/MyWiki.setup`. Under the git plugin settings, find the line
defining 'git_wrapper'. It should look more or less like this:

	# git plugin
	# git hook to generate
	git_wrapper: /home/ikiwiki/MyWiki.git/hooks/post-update
	# shell command for git_wrapper to run, in the background
	#git_wrapper_background_command: git push github

Since the bare repository will not be pushing to ikiwiki anymore, disable the
git_wrapper by prepending a '#' sign to its line (this will turn it into a
comment).

Now, tell ikiwiki to push to the 'wiki' repository when a change is done via the
web interface by uncommenting the `git_wrapper_backgroud_command` line and
setting the command to `git push`. This section of the file should now look like
this:

	# git plugin
	# git hook to generate
	#git_wrapper: /home/ikiwiki/MyWiki.git/hooks/post-update
	# shell command for git_wrapper to run, in the background
	git_wrapper_background_command: git push

In order to tell ikiwiki to listen to pings from the git server and rebuild the
web pages, enable the 'pingee' plugin by adding it under the add_plugins
variable. It should look more or less like this:

	add_plugins:
	- goodstuff
	- websetup
	- pingee

Now rebuild the wiki, so it takes the new setup into account:

	$ ikiwiki --setup MyWiki.setup --rebuild --verbose

Once side of our new setup now should work: Whenever a change is made to the
wiki via the web interface, it should push it to the git server. You can test it
by making a change, and then pulling in your personal clone and make sure you
received the change. The other direction doesn't work yet: Ikiwiki can pull, but
it doesn't know when. We are going to tell the git server to ping ikiwiki
whenever a change is made via a personal clone (i.e. not the web interface).

On the git server machine, as the git user, go to the hooks folder of the 'wiki'
repository. A typical path would be `/home/git/repositories/wiki.git/hooks`.
Create a new file named `post-receive`. This file is a script which will be
executed whenever a new commit it received. The first line should be the shebang
`!#/bin/sh`, then you can add a comment (a comment line starts with `#`) and
finally the command itself. We'll need to tell the git server to send a ping to
ikiwiki.

If you use I2P or Tor, you need to tell the git server to use a proxy to access
ikiwiki, because otherwise it cannot access .onion websites or .i2p eepsites. If
you use the typical port 4444 as the I2P HTTP proxy, you can set the proxy like
this:

	export http_proxy="http://localhost:4444/"

If you use privoxy (I use it), specify the privoxy port instead (usualy 8118):

	export http_proxy="http://localhost:8118/"

If you use the clearnet (regular internet), you don't need the proxy.

Now, the command itself. Basically, we'll use a call to 'wget' (time to install
it if you haven't yet) which will trigger the ping, and direct the returned
result to /dev/null because we don't need it - when you push from a personal
clone, you'll still see the feedback from the git server, so you'll know the
ping was sent successfully. The wget invocation looks like this, with the URL
being the public URL of the ikiwiki instance:

	wget "http://partager.i2p/ikiwiki.cgi?do=ping" -O /dev/null

However, this is not enough: We need to ping ikiwiki only if the commit was made
from somewhere else, otherwise we will cause ikiwiki to pull again, while the
git server isn't done updating its own repository yet with the commit sent by
ikiwiki. There are two ways to check where the commit came from:

- When a commit is made via the web, the author email ends with '@web' or is the
  ikiwiki user itself
- If you use Gitolite, it tells you the name of the committer user via the
  'GL_USER' variable

The second option is better if you use gitolite, because the first one has a
corner case where it may not work: If you push commits you created on a local
ikiwiki clone via local CGI, the author email would contain '@web' as well. If
this use case is relevant to you, you can try to cross the author email with the
committer email. Maybe it will then cover this case - I didn't try.

For the first case, the author test looks like this:

	git log -1 --format=format:%ae HEAD | grep -e '@web$' -e 'ikiwiki@HOST'

Just change HOST to the hostname of the machine where ikiwiki runs. The
'ikiwiki' before the @ is the name of the ikiwiki user we created. To be sure,
you can check the author of the first commit made to your wiki. This should be
the ikiwiki user. You can determine this by running `git log --format=format:%ae
HEAD` on any wiki clone (or on the 'wiki' bare repo itself) and looking at the
last email in the list.

For the second case, the test looks like this:

	[ x"$GL_USER" = x"ikiwiki" ]

"ikiwiki" here is the name of the Gitolie-registered user who made the commit.
The user name comes from the filename of the SSH key. For example, if you added
ikiwiki's SSH key to Gitolite as 'ikiwiki.pub', the username to test here is
"ikiwiki".

Now use `||` to combine the test and the ping: If the commit does *not* come
from ikiwiki, then send the ping. Using the first case, it looks like this
(again, replace HOST with the correct hostname):

	git log -1 --format=format:%ae HEAD | grep -e '@web$' -e 'ikiwiki@HOST' || \
	wget "http://partager.i2p/ikiwiki.cgi?do=ping" -O /dev/null

And for the second case it looks like this:

	[ x"$GL_USER" = x"ikiwiki" ] || wget "http://partager.i2p/ikiwiki.cgi?do=ping" -O /dev/null

Here's a full example, using Gitolite (second case) and an I2P address for
pinging accessed via Privoxy:

	#!/bin/sh
	
	## special hook for ikiwiki repo:
	## update the srcdir clone (which will regenerate HTML),
	## if a commit not-from-ikiwiki was received
	## (if it's from ikiwiki, it already has it so no need
	##  to update it... that would cause a deadlock)
	
	## "ikiwiki" here is the gitolite-registered user (i.e.
	## username determined by SSH key name, e.g. ikiwiki.pub)
	## from which the push came
	
	export http_proxy="http://localhost:8118/"
	[ x"$GL_USER" = x"ikiwiki" ] || wget "http://partager.i2p/ikiwiki.cgi?do=ping" -O /dev/null

Save the file. git won't use it unless it is executable, so we need to update
permissions. These are the command for the last step, and the last command is
the permission change (makes our hook executable):

	# su - git
	$ cd repositories/wiki.git/hooks
	$ nano post-receive
	$ chmod 750 post-receive

Time to test the new setup. Using a personal clone, push change and make sure
the HTML gets updated automatically. Then, just to be sure, you can try again to
make a web edit and make sure it got pushed by pulling in my personal clone and
seeing the update.

Done. You can delete the old bare repository ikiwiki created, to avoid confusion
later:

	# su - ikiwiki
	$ rm -rf MyWiki.git

Congratulations! You have a new working ikiwiki!