This documentation is for Dovecot v1.x, see wiki2 for v2.x documentation.

Solr Full Text Search Indexing

Solr is a Lucene indexing server. Dovecot communicates to it using HTTP/XML queries. Supported since Dovecot v1.1.2 and later.

Compiling

Give --with-solr parameter to configure script. You'll also need to have libexpat and libcurl installed.

Configuration

Replace Solr's existing solr/conf/schema.xml using doc/solr-schema.xml from Dovecot. You may want to check if the file contains something you want to modify. See Solr wiki for how to configure it.

On Dovecot's side add:

protocol imap {
..
  mail_plugins = fts fts_solr
}
# POP3 doesn't do any FTS lookups or indexing, but indexed messages may be deleted
protocol pop3 {
..
  mail_plugins = fts fts_solr
}

plugin {
  fts = solr
  fts_solr = url=http://solr.example.org:8983/solr/
}

Fields listed in fts_solr plugin setting are space separated. They can contain:

Solr optimization

Solr indexes should be optimized once in a while to make searches faster and to remove space used by deleted mails. Dovecot never asks Solr to optimize, so you should do this yourself. Perhaps a cronjob that sends the optimize-command to Solr every n hours.

Sorting by relevancy

Solr/Lucene supports returning a relevancy score for search results. If you want to sort the search results by the score, use Dovecot's non-standard X-SCORE sort key:

1 SORT (X-SCORE) UTF-8 <search parameters>

Indexes

Dovecot creates the following fields:

Lucene does duplicate suppression based on the "id" field, so even if Dovecot sends the same message multiple times to Solr it gets indexed only once. This might happen currently if multiple searches are started at the same time.

You might want to build a cronjob to go through the Lucene indexes once in a while to delete indexed messages (or entire mailboxes) that no longer exist on the filesystem. It shouldn't normally find any such messages though.

Sharding

If you have more users than fit into a single Solr box, you can split users off to different servers. A couple of different ways you could do it are:

External Tutorials

External sites with tutorials on using Solr under Dovecot

Plugins/FTS/Solr (last edited 2010-09-27 14:13:51 by TimoSirainen)