Rel4tion-Wiki/maint/admin/mail.mdwn

I'm running a mail server here, at home. So far (May 2015) I'm the only user,
but I welcome nice people who look for a somewhat safe and friendly place to
store their e-mail, which also hopefully promotes decentralization by being a
small home server and not an privacy-abusing corporate giant.

This pages describes how I configure(d) the mail server, in a format of a
guide. It helps me remember what to do because I touch the config once in a few
months, and I hope it can help others with their configuration or with
launching similar servers.

I probably didn't list here every single change I made, and being written as a
guide, the details here aren't exactly identical to my configuration. So if you
have problems or find inaccuracies or mistakes here, please contact me.

Details, features and plans:

- [[!template id=rmtask done=yes text="Dovecot"]]
- [[!template id=rmtask done=yes text="Exim"]]
- [[!template id=rmtask done=yes text="SSL"]]
- [[!template id=rmtask          text="DKIM"]]
- [[!template id=rmtask          text="DNS"]]
- [[!template id=rmtask          text="Linux users"]]
- [[!template id=rmtask done=yes text="Server side sieve mail filtering"]]
- [[!template id=rmtask prog=yes text="Server side POP and RSS aggregation"]]
- [[!template id=rmtask prog=yes text="Calendar and contacts on the server"]]

Contents:

[[!toc]]

# Basic Mail Server

TODO explain DNS records (MX, etc.)

	# apt-get install exim4-daemon-light exim4-config

Installs packages usual.

	# dpkg-reconfigure exim4-config

Shows config UI.

- internet site
- rel4tion.org
- empty
- In the other domains list, add any domains you wish to serve (in my case it's
  just `rel4tion.org` right now, but I did also use I2P and OpenNIC ones until
  recently).
- If you don't need relaying, leave the relay domain field blank.
- Same for smarthost IP addresses.
- For Dial-on-Demand, you probably want to choose No
- In the mail format screen, I chose "mbox". It seems like a safe default, and
  as the description says - other tools usually expect mbox. Also, "Maildir" is
  in English, and I want to be able to localize folder names (in my case to
  Hebrew). The only "drawback" with mbox is that each folder can either contain
  folders, or contain messages. Never a mix of both. No big deal.
- I chose not to split the configuration and have a single file. You can read
  about it in Debian's README for the Exim package.

Done, it restarts Exim.

Now the SSL support and other stuff. `/etc/exim4/exim4.conf.template`.

- Place `mail.rel4tion.org.crt` and `mail.rel4tion.org.key` under `/etc/exim4`,
  users `root:Debian-exim` and permission 640
- Create file `/etc/exim4/exim4.conf.localmacros`:

	MAIN_TLS_ENABLE      = true
	MAIN_TLS_CERTIFICATE = /etc/exim4/mail.rel4tion.org.crt
	MAIN_TLS_PRIVATEKEY  = /etc/exim4/mail.rel4tion.org.key

In the config example section at the end of the main config file (remember I
chose non-split), uncomment the `plain_server` and `login_server`.

NOTE: the debian README coming with the exim package is amazing. Read it.

Create file `/etc/exim4/passwd` root:Debian-exim 640. These are users and
passwords that can be authenticated by Exim and send mail remotely. See manpage
`exim4_passwd`.

Have lines of the form `username:encrypted_password`. I make the password using
mkpasswd from whois package). Manpage suggests not to use MD5. The SHA
vairiants seem like good candidates.

To enable port 587 too, put this in the main exim config file:

	daemon_smtp_ports = 25 : 587

Now dovecot.

	# apt-get install dovecot-core dovecot-imapd dovecot-pop3d

Refuse self-signed cert - I use my own. Then it just installs.

In `/etc/dovecot/dovecot.conf`, uncomment:

	listen = *, ::

In `/etc/dovecot/conf.d/10-auth.conf`:

	disable_plaintext_auth = yes
	
	auth_mechanisms = plain login
	
	#!include auth-system.conf.ext
	
	!include auth-passwdfile.conf.ext

In `/etc/dovecot/conf.d/auth-passwdfile.conf.ext`:

	passdb {
	  driver = passwd-file
	  args = scheme=SHA512-CRYPT username_format=%u /etc/dovecot/users
	}
	
	userdb {
	  driver = passwd-file
	  args = username_format=%u /etc/dovecot/users
	}

	mail_privileged_group = mail

Now put users in `/etc/dovecot/users`. The passwords can probably be identical
to the ones in Exim (but need to generate using `doveadm pw -s SHA512-CRYPT`),
but it also needs to specify uid, gid and homedir for each user (can specify
default values in the file we just edited).

The full format is:

	user:password:uid:gid:(gecos):home:(shell):extra_fields

For us what remains is:

	user:password:uid:gid::home

In `/etc/dovecot/conf.d/10-ssl.conf`, enable SSL and set paths of cert and key.

Make sure ports 143 (IMAP), 110 (POP) and 25 (SMTP) are not blocked (also 587
if you use it).

# Mail Filtering

Many users have a folder hierarchy in their inbox, and they drop messages into
folders according to their topic, sender and so on. When you get a lot of
e-mail or have many folders, this manual filtering work becomes too hard and
you can ask the computer to do it for you, using filtering rules.

There are two places filtering can happen: Client side (in your e-mail
application) and server side (in the remote computer which receives your
messages for you).

Many e-mail clients have a filtering feature. You easily define rules using a
GUI. As long as you read your e-mail from one computer, it works well, but what
if you want to check e-mail from some other computer, e.g. a public one? Your
filtering rules aren't there. Messages start coming and filling your main
inbox. You want the automatic filtering to clean the mess, but it will happen
only when you come back home and launch the client there, where the rules are
defined.

Letting the server filter the messages means you get the same experience,
independently of the client! Even as you sleep, your mail server receives
messages sent to you and puts them in the right folders. In the morning, you
can open your e-mail client and everything is exactly the way you like it. And
it works even when you read mail from your Replicant phone or a friend's
computer.

Install LMTP support for Dovecot:

	# apt-get install dovecot-lmtpd

Update the Exim router to use LMTP instead of local delivery:

	local_user:
	  debug_print = "R: local_user for $local_part@$domain"
	  driver = accept
	  domains = +local_domains
	  check_local_user
	  local_parts = ! root
	  #transport = LOCAL_DELIVERY
	  transport = dovecot_lmtp
	  cannot_route_message = Unknown user

Add an Exim transport for LMTP:

	dovecot_lmtp:
	  driver = lmtp
	  socket = /var/run/dovecot/lmtp
	  #maximum number of deliveries per batch, default 1
	  batch_max = 200

Add `acl_smtp_rcpt` rule that denies recipients not listed in Dovecot:

	  # Deny recipients that don't exist in Dovecot
	  deny
	    message = invalid recipient
	    domains = +local_domains
	    !verify = recipient/callout=no_cache

Update Dovecot's `auth-passwordfile.conf.ext` to accept `user@domain`
usernames. Since this is what Exim does, it seems to be required for LMTP to
work. Just change `%u` to `%n`:

	passdb {
	  driver = passwd-file
	  args = scheme=SHA512-CRYPT username_format=%n /etc/dovecot/users
	}
	
	userdb {
	  driver = passwd-file
	  args = username_format=%n /etc/dovecot/users
	}

Update Dovecot's `10-mail` config file to use just the local part in inbox file
paths: (`%n` instead of `%u`)

	mail_location = mbox:/var/mail-dirs/%n:INBOX=/var/mail/%n

Now restart Dovecot and Exim, try sending an e-mail from the server and to the
server. Works, in both directions? Great. If not, feel free to ask me! The IRC
channels of Exim and Dovecot seems not to be very responsive or helpful, or
maybe it's just my badly phrased questions, but anyway ping me on IRC.

The logs of Exim (`/var/log/exim4/mainlog`) and Dovecot (`/var/log/mail.log`)
will help you troubleshoot problems.

Server side filtering uses the Sieve language (for defining filtering rules)
and the MANAGESIEVE protocol (for editing filtering rules remotely). Install
required packages:

	# apt-get install dovecot-sieve dovecot-managesieved

Enable sieve plugin for LMTP (`20-lmtp`):

	protocol lmtp {
	  mail_plugins = $mail_plugins sieve
	}

Prepare directory for sieve scripts:

	# mkdir -m 770 /var/mail-sieve
	# chown mail: /var/mail-sieve

Configure sieve in `90-sieve` (unmodified parts omitted for clarity):

	plugin {
	  # The path to the user's main active script. If ManageSieve is used, this the
	  # location of the symbolic link controlled by ManageSieve.
	  #sieve = ~/.dovecot.sieve
	  sieve = /var/mail-sieve/%n/active.sieve
	
	  # Directory for :personal include scripts for the include extension. This
	  # is also where the ManageSieve service stores the user's scripts.
	  #sieve_dir = ~/sieve
	  sieve_dir = /var/mail-sieve/%n/scripts
	}

The Dovecot wiki says that `sieve_dir` is deprecated, and it can be specified
inside the `sieve` variable. But Trisquel's default config (inherited from
Debian, I imagine) uses `sieve_dir`, so it does work.

Enable the managesieve servics in `20-managesieve`:

	service managesieve-login {
	  #inet_listener sieve {
	  #  port = 4190
	  #}
	
	  #inet_listener sieve_deprecated {
	  #  port = 2000
	  #}
	
	  # Number of connections to handle before starting a new process. Typically
	  # the only useful values are 0 (unlimited) or 1. 1 is more secure, but 0
	  # is faster. <doc/wiki/LoginProcess.txt>
	  #service_count = 1
	
	  # Number of processes to always keep waiting for more connections.
	  #process_min_avail = 0
	
	  # If you set service_count=0, you probably need to grow this.
	  #vsz_limit = 64M
	}
	
	service managesieve {
	  # Max. number of ManageSieve processes (connections)
	  #process_limit = 1024
	}

Time to test. I expect a permission problem with creating the per-user files
and folders under `/var/mail-sieve`. May require creating them manually, or
fixing permissions. Use the logs to find error causes.

How to test? So far I mostly see GUI and web clients which support managesieve,
and none of them is what I use. If you use such a client, go ahead and try. For
the command like, you can try to:

	# apt-get install sieve-connect

Usage example:

	$ sieve-connect --server mail.rel4tion.org --debug --4 --port 4190

You can also use telnet. The Dovecot wiki explains the commands, it's quite
simple to test.

A minimal sieve script to test with is one which keeps every message in the
inbox, i.e. the same as having no filtering at all. It's a single line:

	keep;

# POP Aggregation

If you, as a user, have several email accounts, you can collect all the
messages from all of them into your local mail server. It can be done by
downloading e-mail from these accounts using the POP protocol, and deleting it
from the remote servers.

TODO explain better how it works?

This server uses *mpop* for this purpose. I prepared a simple setup which uses
*mpop* and *msmtp*, to make things easier. See [[!rel4git doar]]. Check out the
README there. Basically it allows system users to define a list of mail
accounts, and a cron job runs through these lists and collects messages into
users' mailboxes.

# RSS Aggregation

This server uses *rss2email*, which allows system users to define a list of RSS
feeds. Then you can use a cron job to collect the users' feeds and send them as
email messages. *rss2email* is easy to use and there's good documentation.

TODO explain it anyway?

# Calendar and Contacts

Two groupware features commonly used with e-mail are calendars (with events and
reminders etc.) and contact lists. With these available on the server, you can
access the calendar and contacts remotely, share calendar items and so on.

The technologies this server uses for this are [[!wikipedia CalDAV]] and
[[!wikipedia CardDAV]], for calendars and contacts respectively.

The server software used is *DAViCal*. There's also a very simple and
lightweight server named [[calypso|http://keithp.com/Calypso]], but it's made
for a single user, while what we need here is community server support (i.e.
should be able to host several users).

DAViCal uses PHP and PostgreSQL. This server already has PostgreSQL for the
MediaGoblin instance, and uses the Lighttpd web server. So the following
installations are needed:

	# apt-get install davical

TODO continue