netcon-source / heliotrope Goto Github PK

Heliotrope is a personal email server. It provides all the functionality you
want in a modern email client:

- proper message threading
- labels
- fast, full-text search over all messages with a complete query language
- support for signed and encrypted email
- an extensible JSON-over-HTTP API

Heliotrope is a backend service against which email clients / MUAs can be
written. To use it, you must use a client. For an example client, see
Turnsole: http://github.com/wmorgan/turnsole.

WHY ANOTHER PROTOCOL? WHY NOT JUST USE IMAP?

Because IMAP is terrible and you want all those features listed above.

REQUIREMENTS

Heliotrope is written in a mixture of Ruby and C. You will need a working C
compiler, and a modern Ruby. 1.8.7 or 1.9.2 are good choices. Ruby 1.9.2
will give you better performance and better i18n support.

Heliotrope uses html2text to generate text from HTML attachments. You will
need this or some reasonable alternative.

PREPARING YOURSELF

0. Find your happy place.
1. Install html2text. On Debian-based Linux distros, it's `apt-get
   install html2text`. On OS X, `port install html2text` and `brew install
   html2text` have both been known to work.
2. Install a modern Ruby. I recommend 1.9.3. On Debian-based Linux distros,
   you can try the ruby1.9.1 package. You can also download and compile it from
   http://www.ruby-lang.org/en/downloads/. For Debian-based Linux distros,
   make sure you have the libreadline-dev and zlib1g-dev packages installed.
3. Run `gem install trollop whistlepig rest-client sinatra rmail leveldb-ruby locale lrucache curb`

GETTING IT

Once Heliotrope is more stable, you will be able to install it via rubygems.
For you, you should run this:

  git clone https://github.com/wmorgan/heliotrope.git

RUNNING IT

Decide where you want to store everything. Call this directory <mailstore>.
Now run `ruby -Ilib bin/heliotrope-server -d <mailstore>`.

Heliotrope exposes a simple debugging HTML interface. If you point your
browser to http://localhost:8042/, you should see your empty mailstore.
Congrats! You're running Heliotrope.

IMPORTING EXISTING MAIL STORES

To bulk import mail, use heltrope-import. You must stop your server first.

To add a pre-existing mbox:
  ruby -Ilib bin/heliotrope-import -m <mbox> -d <mailstore>

To add a pre-existing maildir:
  ruby -Ilib bin/heliotrope-import -a <maildir> -d <mailstore>

To add a pre-existing Gmail account:
  ruby -Ilib bin/heliotrope-import -g <gmail username> -d <mailstore>

To add a pre-existing, non-Gmail IMAP account:
  ruby -Ilib bin/heliotrope-import -i <imap server> -d <mailstore>

For testing purposes, you may want to limit the number of emails added by
using the --num-messages option. If you additionally use the --state-file
option to save state, successive invocations of heliotrope-import will
resume where the stopped.

After import, start the server again. You should see stuff at
http://localhost:8042/.

For a full client, see https://github.com/wmorgan/turnsole/.

ADDING EMAIL

Once the server is running, you can use heliotrope-add to add individual
emails as they arrive. Unlike heliotrope-import, heliotrope-add talks to
your running server.

To add a single email:
  cat email.txt | ruby -Ilib bin/heliotrope-add

To add messages from existing GMail, IMAP, mbox or maildir sources, use the
same arguments as to heliotrope-import above. Note that if you specify
--state-file, every run of heliotrope-add will pull in any new messages from
the source. This makes it easy to mirror an existing account.

For example, the following bash script:

  while true; do
    ruby -Ilib bin/heliotrope-import -g <gmail username> -d <mailstore>
    sleep 300
  done

will mirror mail from a GMail account into Heliotrope with a 5-minute poll
time.

JSON API SPEC
-------------

Heliotrope features an complete JSON-over-HTTP API. Documentation coming
soon! For now, you can reverse engineer lib/heliotrope-client.rb.