searching e-mails with wanderlust and mu

I have discussed the Wanderlust e-mail client a couple of times already. I'm
still using it, so I keep on learning new tricks. Even though there has been
quite a bit of action in the competing gnus e-mail client, for my particular
use-case, Wanderlust is still the best option.

'My particular use-case' consists of storing my mail in Maildirs, which I fill
with either offlineimap (which has fortunately found a new maintainer) or
fetchmail.

mu

When dealing with e-mail, one particularly important feature for me is the
ability to search my messages. In fact, it's so important for me that I
wrote some software to do this for me; the software is called mu; it indexes
the messages in my Maildirs, and then allows for searching them using
queries, based on message contents, headers, or other message properties.

mu works through a command-line interface, although there is an
experimental GUI available as well. The command-line interface makes it
possible to hook mu up with various mail-clients, such as mutt, or
Wanderlust. Some Linux distributions ship mu, but since the versions they
ship are often a bit outdated, I recommend building it yourself from the
sources linked on the mu website. The process is fairly straightforward;
and there is plenty of documentation in the form of man pages.

mu and wanderlust

I've been combining mu and wanderlust for a while (see mu and wanderlust - the old way, below), but this week Sam B. on the mu mailing
list showed a way to do so in a much more elegant way - using virtual or
query folders.

How does this work? Well, after installing mu, add the following to your
Wanderlust setup file (~/.wl or it's moral equivalent – see the older
Wanderlust posts for the details):

(require 'elmo-search)
(elmo-search-register-engine
    'mu 'local-file
    :prog "/usr/local/bin/mu" ;; or wherever you've installed it
    :args '("find" pattern "--fields" "l") :charset 'utf-8)

(setq elmo-search-default-engine 'mu)
;; for when you type "g" in folder or summary.
(setq wl-default-spec "[")

So, to start with the last part, whenever you type g in folder or summary,
in the mode-line you will get something like Folder name (.inbox): [. Now
simply type your mu search expression and press Enter, and wanderlust
opens a (temporary) folder with the search results. Brilliant!

Next, to add virtual folders for searches you do often, simply add some
folder specifications like the following to your .folders file (again,
check the older Wanderlust posts if you're not familiar with folders-file):

VFolders {
# message I received today
  [date:today..now]!mu  "Today"

# messages bigger than 1Mb  
  [size:1m..100m]!mu    "Big"

# signed messages i got in 2010 related to emacs
  [date:2010..2011 flag:signed emacs]!mu "Signed-Emacs2010"

# unread messages
  [not flag:seen]!mu    "Unread"
# or (for mu  >= 0.9.4):
# [flag:unread]! mu      "Unread"
}

After this, restart Wanderlust, and there you go! Wanderlust will display
your brand new virtual folders with an icon that looks like a little whale.

You can put arbitrary mu search expressions between the [], matching
whatever is useful in a certain case. Check the mu documentation to see how
to do this.

Note, the messages you get in these virtual folders are links to the
original messages. In practice, this means that changes you make to the links
do no affect the originals – if you delete a link you're not deleting the
message.

mu and wanderlust - the old way

This discussion would not complete without a description of the old way I
used search. This method may still be useful for integrating mu with other
clients such as mutt.

What I've been using for a while is a (in retrospect) rather clumsy way to
integrate message searches with Wanderlust: based on the results of a query,
I would create some special Maildir and fill it with symbolic links to the
matched messages, and the visit this special Maildir with Wanderlust. I'll
include the code here to contrast it with the more elegant solution that we
saw before, but also because the approach taken might be easily adapted for
other mail-clients.

;; search using mutt
(defvar mu-wl-mu-program     "/usr/local/bin/mu")
(defvar mu-wl-search-folder  "search")

(defun mu-wl-search ()
  "search for messages with `mu', and jump to the results"
   (let* ((muexpr (read-string "Find messages matching: "))
          (sfldr  (concat elmo-maildir-folder-path "/"
                    mu-wl-search-folder))
          (cmdline (concat mu-wl-mu-program " find "
                      "--clearlinks --format=links --linksdir='" sfldr "' "
                     muexpr))    
          (rv (shell-command cmdline)))
    (cond
      ((= rv 0)  (message "Query succeeded"))
      ((= rv 2)  (message "No matches found"))
      (t (message "Error running query")))
  (= rv 0)))

(defun mu-wl-search-and-goto ()
  "search and jump to the folder with the results"
  (interactive)
  (when (mu-wl-search)
    (wl-summary-goto-folder-subr
      (concat "." mu-wl-search-folder)
      'force-update nil nil t)
    (wl-summary-sort-by-date)))

;; search by pressing 'Q'
(define-key wl-summary-mode-map (kbd "Q") ;; => query
  '(lambda()(interactive)(mu-wl-search-and-goto))) 
(define-key wl-folder-mode-map (kbd "Q") ;; => query
  '(lambda()(interactive)(mu-wl-search-and-goto))) 

After installing mu and putting the above in your wanderlust startup
file, you should be able to search by pressing Q. The mu documentation
has an example for mutt as well.

conclusion

It's straightforward to integrate advanced searching capabilities to
Wanderlust using mu, and thanks to Sam B., it's gotten a lot easier! The
second (old) approach may be useful as 'inspiration' for use in other e-mail
clients as well, if they do not provide the kind of hooks that the first
solution needs.