Taking email offline

This week I’ve got a bee in my bonnet over online tracking and data mining. I want less of it. I’m not taking a hard line—I don’t plan to delete google and facebook entirely—I just want to take back some control of my data. For example, I installed the AdBlock and Disconnect plugins for Firefox to cut down on surreptitious tracking while I surf the web, and I switched my default search engine to DuckDuckGo. I also spent a lot of time thinking about how to replace gmail as my primary email address.

Considerations

If you use gmail, google scans the contents of your messages so they can serve you targeted advertisements. And when you’re logged in to gmail they can more easily track your web browsing for data mining and to serve more targeted advertisements (although the plugins mentioned above should help with that). I don’t think google is evil or anything, I just don’t want to pay for their “free” service with my data any more. I’d rather pay a few bucks a month to keep my emails to myself, and as a bonus I’ll no longer be logged in to google all day.

Now, I love gmail’s interface. I’m a person who never bothers to save messages to folders, so I have an inbox with 10,000 messages and I use search to find what I need. Whatever setup I used to replace gmail would need to have excellent search capabilities. Also, I want to keep using my gmail account to sign up for things on the internet (my new non-gmail email will be used for correspondence), so I want a client that let’s me easily view mail from both accounts. And even though I’m planning to use a non-web-based client on my personal laptop, it would be nice if my new email had a shiny web interface for when I’m out and about.

I considered paying for a fastmail account. They appear to offer a great service, and their recently launched calendar app could replace google calendar for me. But in the end it made more sense to use a service I’m already paying for: hosted email from DreamHost. Sometime soon I’ll try to install ownCloud as a google calendar and dropbox replacement.

The setup

I’ve converged on the following setup:

• OfflineImap to sync my mail accounts to local a local maildir
• mu for indexing and searching mail
• mu4e mail client (with html2text to display html mail)
• msmtp for sending mail

OfflineImap creates a local copy of your mail archive and keeps it in sync with your mail server. I’m using the Maildir format for my local copy, which just means that every email is a file in a particular folder. This is cool, because I can operate on my mail archive using posix utilities. If I rm a file from my Maildir then OfflineImap will delete the message from server. With OfflineImap I have access to all my email when I’m not connected to the web and my mail client will never hang during sync.

mu is my replacement for gmail search. Built on the xapian search engine library, mu indexes my messages as they come in and provides a friendly search interface. The query syntax is straightforward, and you can search header and date fields in addition to message content. mu4e is a mail reader built around mu as a set of emacs modes. It’s a text-based interface, familiar if you’ve used pine or mutt for email (and probably offputting if you haven’t). It provides hooks to bookmark predefined searches and to batch mark messages for deletion or flagging. There’s a nice shortcut to further narrow the currently displayed search. Following a suggestion in the docs, I installed html2text, which provides a better view of html emails that the equivalent emacs mode.

Another program that will index and search your Maildir is notmuch, which I also played around with. The notmuch interface is similar to mu’s (it’s also built on xapian). There are philosophical differences—notmuch provides functions to apply arbitrary tags for email, while mu prefers not to rely on any metadata not contained in the messages themselves. I mostly chose to use mu because I thought the documentation was better.

msmtp is a small program that connects to your mail server to send email. I’m actually not entirely clear on the advantages of msmtp over built-in emacs functionality, but it kept coming up in people’s configurations that I read about, and I decided to follow the crowd. One possible advantage, if I’ve read correctly, is that mstmp makes it easier to send mail through multiple accounts as needed. I haven’t tried this out.

A couple more miscellaneous notes. I spent a while trying to figure out how to run a spam filter locally after OfflineImap syncs; I was surprised that this was nontrivial. The problem seems to be that filters (like spamassassin) are usually called by procmail when mail is “delivered”, and OfflineImap’s operation doesn’t “deliver” mail in the required sense. I found some possible workarounds, but I decided that the reasonable solution was to use the provided server-side tools. I selected roundcube to be my web-based mail client for when I’m checking messages from my laptop; it looks a heck of a lot nicer than squirrelmail.

We’ll see how this goes.

Update: In a follow-up post I go into more detail about how I configured these programs.