Categories
josephscott

Email Tags

I’m just going to come out and say it: I want the ability to tag my email. I know, I know it sounds like I’m jumping on the tagging bandwagon. So what, I want to be able to tag my email. I really only have one reason for this request, to make email search better.

More and more searching for emails has become better at finding old information than the traditional hierarchy method. Sure Gmail, Thunderbird and Mail on Mac OS X have popularized this idea, but it goes beyond that. With the desktop search features heating up, most of them supporting searching email stored on your system, we need to have betters ways for search engines to rank our data. This is about finding the information you need as quickly and easily as possible.

Another point I want to make clear, I’m not interested in pushing email tagging as a replacement for hierarchy storage of email. If you want to go all the way and have one big archive folder with lots of tagged emails, fine. If you want to create a folder tree for you email, that is fine too. Tags should be used to enhance what were are doing, not necessarily replace it.

How about some nuts and bolts stuff, how should email tagging work?

Where should email tags be stored?

My first thought on this is in the headers. So far I still think this is the best place for them. When an email client adds or removes a tag the headers get changed accordingly and the message gets updated on the server (if they are using something like IMAP). I’m not sure what format would be best for these tags, for now I’d envision something as simple CSV (comma separated values). This would allow for spaces, but probably not newlines. I think embedded newlines in the headers would break things.

We’d have to come up with a new header, perhaps x-tags?

What email clients could do with tags.

Once we get tag support in email clients there are a couple of things I’d like to see. The obvious one is the ability to add, remove and edit tags for a given email. It should keep a running list of tags you’ve already used so that you can pick from a list. Yes, it could even generate a tag cloud if you like. A plus would be the ability to suggest tags based on content of the email and how you’ve tagged similar emails in the past. Maybe an autotag feature, that looks at the content of the email for words and terms that match tags you’ve used in the past and selects those for you. From there you could then refine what tags you want assign to the email. For bonus points I’d like Procmail to have some these abilities too, that way your email server could do some basic tagging for you.

Email clients should provide some additional smart folders also. A smart folder called tags could have subfolders that correspond to each tag you’ve ever used. You should be able to sort these alphabetically and by frequency (most commonly used tags at the top or bottom). Bonus points for the ability to drag and drop these smart tag subfolders to do searches. So dragging the ‘work’ tag subfolder onto the ‘sql’ tag subfolder should automatically do start a new tag search subfolder that contained only emails that have those tags. By dragging the ‘mysql’ tag subfolder onto it, the search should be refined even further.

When performing a free form search in email clients any tag matches should get increased importance. If I happen to use the word ‘oracle’ in a search and it is also one of my email tags, then emails with that tag should be ranked higher than emails without it.

How to make this happen.

I suppose someone should write an RFC on email tags for starters. That will outline the technical issues involved and allow vendors to start commenting and working on implementing these features. It would be really great to see someone like GMail support this because once they turn this feature on then everyone will have access to it. It also isn’t that far away from their label feature. Other good targets would include open source projects like Thunderbird and Squirrelmail.

It’s 2005, I hope that I don’t have to wait until 2010 before email tagging becomes widely available.

23 replies on “Email Tags”

Check out Alexey Melnikov’s Common IMAP keywords (draft-melnikov-imap-keywords-03), which is

“to document some common keywords for the purpose of improving interoperability between different IMAP mail clients. The document both documents some keywords already in use, as well as introduces several new ones.�

It was published the same day you posted this blog item (2005-Aug-22)! Details are here:

http://www.ietf.org/internet-drafts/draft-melnikov-imap-keywords-03.txt

Nancy
also dreaming about tagging my email

Just out of curiosity, is tagging your email the same thing as setting IMAP keywords? Only that needs no RFC, just a client that allows you to use arbitrary keywords, and “do things” with them later.

I’ve done the first bit in my experimental client, Infotrope Polymer, and I’m working on the second.

To add to the list of handy Internet Drafts, the SIEVE imap4flags extension just went into last call – that allows a server-side filtering engine to set arbitrary keywords and IMAP system flags – or tags, basically – as the mail is delivered. Cyrus IMAP already supports an early version of this draft, so you can do this today. Nice, eh?

I’ve gave the IMAP keywords RFC a quick glance. This doesn’t appear to provide for a generic tagging method that can be stored in the email headers.

I’d like a free form field that I simply add tags ala del.icio.us.

It doesn’t store them in the headers – or at least, if it does, there’s no way to know. There is already a Keywords header, I think, but nobody uses it. In IMAP, the actual message data is static, only the flags can be changed, so a header isn’t the right thing in this case.

But most IMAP servers do allow for arbitrary flags – IMAP calls them keywords, as opposed to system flags. Most IMAP clients are rather poor in taking advantage of this – for instance Mozilla only allows five predefined “labels”, whose meaning is entirely local, unfortunately.

The I-D Nancy pointed out is purely a set of keywords intended to be common between all clients.

My client does allow you to twiddle with all the flags, including keywords, and define new ones. The server maintains the list of the ones you’re using anyway. SIEVE allows some basic keywording as the message arrives.

My client also allows you to search for them, and colour them in the summary – the UI’s terrible right now, but it works. Since it’s all using very common, server-side, features, it all works now, and your “tags” (or keywords, as IMAP has called them since at least 1994) are available everywhere.

There’s also ANNOTATE, which I don’t think is yet implemented anywhere, which allows for much richer metadata management on IMAP.

So in summary, your wishes came true for the server-side in 1994. Your wishes for the client side – well, give me a few more days. 🙂

Storing tags on the IMAP server would be fine, as long as the IMAP client had a way to cache these. Without some sort of client side caching in place I’d expect searching on these keywords to be pretty painful.

One bummer part about storing them on the server would be moving from one IMAP server to another. Unless there was a way to included these server side keywords/tags then they would be lost when moving from one IMAP server to another.

Another negative for server side storage is being able to tag advantage of forwarded email that has been tagged by someone else already. I’m thinking of the situation where I get an email and add a few tags then forward it to a co-worker. That co-worker would then being able to add or remove tags from that fowarded message.

Being able to tag emails is not only helpful to the individual, but also in a greater social context as well. Again, del.icio.us is a good example of this.

>Storing tags on the IMAP server would be fine, as long as the
>IMAP client had a way to cache these.

This is a quality of implementation issue and most clients that support keywords can already cache them.

>Without some sort of client side caching in place I’d expect
>searching on these keywords to be pretty painful.

It depends on IMAP server implementation and how powerful is the machine running it.

>One bummer part about storing them on the server would be moving
>from one IMAP server to another. Unless there was a way to
>included these server side keywords/tags then they would be lost
>when moving from one IMAP server to another.

IMAP preserves keywords when copying between mailboxes on the same server. IMAP allows to specify keywords when they are uploaded to the server. IMAP lets clients to fetch keywords
from the server.
So what you describe is a non issue as existing clients
like Thunderbird and Pine can already preserve keywords when moving mail around.

>Another negative for server side storage is being able to tag
>advantage of forwarded email that has been tagged by someone
>else already. I’m thinking of the situation where I get an
>email and add a few tags then forward it to a co-worker.
>That co-worker would then being able to add or remove tags
>from that fowarded message.

There is currently no standard for forwarding keywords over SMTP, however some servers like Cyrus LMTP support an extension for doing that.

a lot of talk about email tags – has no one tried the Outlook Categories feature?? That’s been around for years.

I knew I wouldn’t be the only one wanting the ability to tag emails. Earlier today I was lamenting that I can’t tag my email, then I found this blog post through a search for email tags.

A few thoughts to add:

Mark mentions Outlook Categories. They are not tags. Categories you have to set up, then select from a list. Tagging allows you to type text freely. Now, for some, a dozen categories will do. Others will use a much broader set of tags and need the implementation to work equally well with 200 different tags.

I expect we’ll see a web based email service like Gmail offering tags before any stand-alone client brings out a decent implementation. A service like Gmail can just index tags in their own way without having to interact with the outside world.

Next some desktop clients will follow suit. Anyone from Opera or Mozilla up for a challenge? As with web-based services, these implementations will probably index tags locally, which leaves out those of use who use IMAP. Personally I’d love to see tagging work interoperably between clients and servers. I store my mail on a server and use IMAP to access it from several clients like Opera, MacOS Mail, Horde IMP (web-based).

But IMAP would be tricky. Many IMAP servers store email in plain text files. If tags are going to be stored within the email header, I expect browsing emails by tag to be sluggish at best. I can’t imagine it working well unless there is an index of tags.

There’s also the question of how your IMAP clients find out about new tags. Scenario: I check my email at work. I’ve already read this message about the next rehearsal, but now I tag it with “theatre” for later reference. Then I go home and check my email again. Only the new messages are retrieved; I have already got the rehearsal one, but my home machine doesn’t know about the new tag. I don’t want to download all my mail again, but I do want a way to update my tags index by syncing with a remote server.

This is a really cool idea and has been buzzing in my mind for a while.
I actually want to try creating a program to create email tag clouds for Outlook 2003.
In Outlook 2003, it’s not true that you have to setup Categories “a priori”. You can actually just add tags as categories, separated by commas.
This should take care of where/how to persist the tags.
All that is needed now is a cool idea for retrieval and most importantly, displaying a tag cloud…
Here is my 2cents worth, hoping that someone will contribute more ideas.

I was going through the somewhat arduous task of sorting my work email today (I have a few thousand messages!), and it occurred to me how great it would be to be able to tag my email. Many of my business emails fall under different categories. If I could tag all the emails I keep, I could more easily pull up emails based the applicable tag, rather than figuring out what folder or sub-folder I put it in. This beats search hands down.
So, I searched online for some solution, and here we are. Nicely put, Joseph. I hope some others catch on to this idea. Especially Thunderbird, my personal fav.

well, we’re working on this. One of the worst things I hate about folders is how you get fragmented islands – IE, Explorer and outlook all use different folders stores. If you have a project that relates to a web page, email, and file – you need to duplicate the folders accross all apps. That sucks so we’re building an app that unifies the folder islands and allows tagging for anything. The key here thought is the interface which does use clouds, but in a way that is much easier to use than delicious.

In general though, I think folders suck. I gave up on filing email long ago when I realized that you can find anything by using sort and type down (type the first few letters of a name to jump to that person) in outlook. I was going to put together a training course for this but got too involved with the product development to continue. It’s based on the principle that email is tagged with so much info as it is, you really don’t need folders. I have over 6000 messages in my PST file and the volume makes little difference when using this method. I have a document that explains how to set up Outlook for this if anyone wants to see it.

Mark

Personal data, that people have actually read, will never get
greater that 4 Gig. Do the math. So why worry so much about
indexing for speed? I have most of my emails since 1996 in two
text files. Inmail.txt (5 Meg) outmail.txt (36 Meg)

I think what you are looking for is MailTags (http://www.indev.ca/MailTags.html)
This is a plug-in for Apple’s Mail.app that offers extensive tagging (keyword, project, notes, due dates, priority) It integrates with Mail’s smart mailboxes (get show all messages with duedate within 3 days) and spotlight.

I am working on version 2.0 which will support saving Tags on IMAP messages so that they are available on multiple computers. Smart Mailboxes will automatically update regardless of which computer you are on and which computer you used to tag the message.

Wow, this is so awesome! I am a soon to be graduate in Information Technology and I have been tossing this email and file tagging/labeling/keywording idea around since I started designing databases in Access almost ten years ago. Databases make indexing and searching a natural way of life to support views and queries, and I thought to myself, “Well, why don’t they make that for files and emails?” Just keep them all in one place (like a table, not in silly folders) and have searches (views or queries) to filter out the files/emails you need with whatever subject or keywords at any given point in time. Then you don’t have to worry where you’re filing crap – or file it at all! My over 10,000 emails at home could use such a system.

I was so happy when I found out GMail and Vista support it. Filing in file folders is thinking in PHYSICAL concepts! It’s about time we thought electronically!

So there’s my two cents. Sorry I can’t contribute to your project personally, but I thank everyone involved…if it’s still going on…

Mark’s idea of using Categories (from the preview pane, highlight the email or emails you want to tag and choose Edit-Categories and assign the Category). You can then sort or search for the messages by Category. I had no idea to try this, but Mark is my hero!

This sounds like a great idea.
I would love to have this as a plugin feature on outlook./
Hey maybe you can even go open source with it and so something for Thunderbird just to get the idea popular.

Taglocity (www.taglocity.com) has a free/commercial Outlook Add-In for tagging. It has some really nice features such as a tag cloud, Bayesian learning for auto-tagging, tag renaming, etc.

I have been testing interoperability between TB’s tagging feature and the Taglocity plugin, and found a somewhat annoying compatibility issue:

TB stores its tags in the “Tags” element of the header, while
Taglocity uses the “Keywords” element! Meaning, of course, that tags aren’t cross-recognized on the two platforms.

Anyone know a fix for this?

@Adam –

Well, at least there are a few options out there now. I’m not sure that they work very well with each other though. There really needs to be a simple email tags spec so that things work as expected.

Leave a Reply

Your email address will not be published. Required fields are marked *