Summary

IMAP (Internet Message Access Protocol) is a widely implemented protocol for email clients to access messages on a server, and allows messages to be stored and filed on a message server. Information on IMAP history, technology and new developments can be found at "The IMAP Connection".

Isode has launched its own IMAP server, M-Box, which also supports POP access. This white paper sets out why this productis an important part of Isode's messaging strategy, and shows how it fits as a part of Isode's overall Internet Messaging Solution.

M-Box is available for evaluation as part of the Isode Internet Messaging Solution for ISPs.

Creative Commons License

Why IMAP is important to Isode

There are a number of reasons why IMAP is important to Isode:

Complete Internet Messaging Solution Isode's strategy is to offer a complete Internet Messaging Server solution, suitable for service providers and other high volume users. Message storage, and IMAP in particular, is a key part of this solution.
Support of Wireless Messaging Isode wishes to provide a server solution which offers good support for Wireless devices using Internet protocols. The functionality defined by the LEMONADE working group of the IETF (Internet Engineering Task Force) is critical to achieve this, and IMAP is a central component of LEMONADE.
Reliable Internet Messaging Isode provides solutions for high assurance messaging. In order to provide a complete reliable messaging solution using Internet email, a managed IMAP server is a key component.
Large Message Archives It is often convenient and desirable to access a message archive from an email client. IMAP is the only standard approach to achieve this.

Approaches to getting email to the end user

There are several ways that an email user can get at email messages. In order to understand the importance of IMAP, it is necessary to understand the different capabilities of the protocols.

Webmail A common way for users to access email is through the Web (e.g., Hotmail, gmail). This approach is referred to as Webmail. Messages are stored on a server which is accessed using the Web. Webmail interfaces will typically provide capabilities to manage and file messages on the Webmail server.
POP High volume email users usually prefer to use desk top email clients (e.g., Microsoft Outlook, Eudora, Thunderbird). The most common way for email clients to access new messages is to use a protocol called POP (Post Office Protocol) to connect to an email server. New messages for a user are delivered to the user's POP server. The email client connects to the POP server and retrieves the waiting messages. Message filing and management is done by the client (e.g., messages are filed on the user's PC).
IMAP

Like POP, IMAP is a protocol to connect an email client to a email server. IMAP can be used to provide the same services as POP. In addition, IMAP is designed to enable messages to remain on the email server, and offers capabilities for remote searching and filing. This allows the client to selectively download messages.

A typical IMAP client will combine local filing and management with IMAP's remote (server) filing and management capabilities.

A detailed analysis of IMAP vs POP is given in Message Access Paradigms and Protocols.

Proprietary Proprietary messaging solutions (e.g., Microsoft Exchange; IBM Notes/Domino) offer proprietary approaches to achieving similar functionality to POP and IMAP.

Isode has no interest in supporting proprietary access mechanisms, and advocates the use of open protocols. It is important to allow mix and match of the three open approaches to email access.

IMAP and Webmail

IMAP can co-exist easily with POP, although an IMAP user will typically not use POP (as you can do everything that POP can do using IMAP). An IMAP user may sometimes use Web access, for example when using a simple Web terminal.

A common and effective way to implement Webmail is as a front end to an IMAP server. This is a good architecture, because an IMAP server can act as a high performance message database with all the functions that are needed by Webmail. This approach will mean that IMAP and Webmail become alternate approaches to access the same messages, which is convenient for both the service provider and the end user.

Trends in Email Access

Webmail, POP and proprietary access mechanisms are heavily used. Although there are many IMAP products and users, it is not used as much as the other mechanisms. Corporate users generally use proprietary email access, and IMAP capabilities are provided with proprietary mechanisms. Service providers generally offer POP and Webmail, and this meets the core needs of most home users and small businesses.

There are a number of email trends which in combination will change this picture.

  1. Multiple email addresses. There is a trend to have at least two email addresses (one for home and one for work).
  2. Always connected culture. Access to home email from work, and to work email from home, and to all email when elsewhere is an increasing priority for many people.
  3. Using more devices for accessing email, including PCs, Laptops and wireless PDAs. Users need a solution which will work for multiple devices.
  4. Need for email on the move (support in wireless devices).
  5. Corporate deployments will generally continue to use proprietary email access.
  6. Service providers, personal email, and small organizations will continue to use open protocols for email access.

Problems for the Mobile User

The reason that these trends will drive IMAP usage is best understood by considering the problems of the mobile user, who needs to access email from many locations.

Low Bandwidth A mobile user will often be connected over slow and expensive network links. Webmail is a poor choice over a slow link, because of the need to keep displaying pages. POP simply downloads messages. IMAP allows sophisticated control over message browsing, searching and download, while managing message display on the client. IMAP is a good approach for dealing with low bandwidth.
Multiple Devices A typical mobile user will use more than one device to read email (laptop; PC;PDA). POP is a problematic with multiple devices, as its download approach can often lead to email being held on the "wrong device" (e.g., it is automatically downloaded to a PC at home, while the user is traveling). IMAP's server-centric approach is much more suitable to handling multiple devices. Webmail is also a good approach.
Disconnected Operation The mobile user is often in a situation where network access is not possible (e.g., on an aircraft). In this situation, Webmail is not viable, and any email you need to process must be downloaded onto an email client. POP has the right behavior for this situation. A good IMAP client, which caches message copies locally, is also a good choice.
Web only access The mobile user will sometimes be in a situation where the only access is via a Web only "Internet terminal". Here, Webmail is the only viable approach.

It can be seen that the mobile user has a complex combination of somewhat conflicting requirements. The best approach for most users is a server-centric solution, with the server supporting both IMAP and Webmail, and used with a clients that supports intelligent caching to support disconnected usage.

LEMONADE and Wireless Devices

Wireless devices, such as mobile phones and PDAs have a number of additional and specific problems:

  1. Form factor. The screen is much smaller than a typical desktop, which generally requires use of a special email client.
  2. Limited storage on the device, which constrains how much email is stored or cached on the device.
  3. Very low bandwidth. Despite plans for high speed mobile networking, mobile devices often need to operate at speeds of 9600 baud and less.

The IETF (Internet Engineering Task Force) has set up the LEMONADE working group to specify protocols to support such devices. The core of the LEMONADE work is based on SMTP (Simple Mail Transfer Protocol) and IMAP, which provide a good basis for supporting wireless devices. There are a number of LEMONADE extensions, to optimize performance and functionality for wireless devices. IMAP and LEMONADE will be critical components for supporting wireless devices.

Message Archives

A common requirement is to hold large numbers of messages, such as messages sent to an email list, in an archive. IMAP is an ideal approach for holding such messages as:

  • A good IMAP server is well optimized for this application.
  • IMAP archives can be accessed with an email client, which is the natural way to examine an email archive.
  • IMAP easily allows message archives to be shared.
  • IMAP allows archive browsing.
  • IMAP allows searching, which is important for a large archive.

IMAP Client Trends

IMAP is very widely supported in email clients, and all of the most popular email clients support IMAP. This wide support is expected to continue. Two developments are anticipated:

  1. Most email clients are optimized for either proprietary access (e.g., Microsoft Outlook) or for POP3 (e.g., Eudora). We expect to see more clients that are optimized for IMAP usage, as IMAP usage grows, and to support advanced features such as message annotation.
  2. Development of LEMONADE conformant IMAP clients for wireless devices.

ISP Requirements for IMAP

Corporate users are expected generally to continue to use proprietary email access mechanisms. The demand for IMAP will come from individuals and home users whose needs are not being met by POP and Webmail. There will be an opportunity for ISPs and other service providers to introduce IMAP services. This needs to be done in a manner which can smoothly co-exist with POP and Webmail, so that it becomes an additional customer choice (perhaps a premium service) that can be used in conjunction with Webmail and POP.

Isode has identified three strategic requirements for a POP3, which are described in more detail below:

  • Vertical scaling.
  • Horizontal scaling.
  • Directory configuration

These server features need to be combined with strong management and monitoring.

Vertical Scaling

An IMAP server needs to go fast, support large numbers of users, and support large numbers of connections. This is what we mean by good vertical scaling. Vertical scaling for an IMAP server is much hard than for POP, as the server needs to handle a large database of messages and perform complex and potentially resource-intensive operations such as searching (a POP user will usually just download all messages to the client). Also, IMAP clients tend to spend much more time connected to the server, and so the server needs to be optimized for large numbers of connections. For a very large system, this means a multi-threaded implementation as a single-threaded implementation leads to too many processes.

Horizontal Scaling

However good the vertical scaling of server is, there comes a point where a service provider will need to operate multiple servers. For many IMAP servers, this becomes a management nightmare, with tools to migrate users between servers, and dealing with configuration issues relating to having different users on different servers. Good horizontal scaling is the ability to easily add new servers.

A different model is becoming popular, often associated with the maildir format. In this approach, all mailboxes are stored on a single NFS server. Companies such as Network Appliances and EMC offer high performance large scale servers that can be used to implement the storage. Having all mailboxes on the same server simplifies mailbox management and migration problems. This approach enables new servers to be added easily in order to share increasing load.

Directory Configuration

Isode's directory vision is that a service provider will have: "One Directory Entry - One Person - One Account". By taking a strong directory-oriented approach, configuration can be shared between multiple IMAP servers and other messaging applications. This is seen as an essential and complementary approach to horizontal server scaling.

Conclusion: Isode and IMAP

Isode has launched its own IMAP server (M-Box) which also supports POP access. We've chosen to build our own server to address the market requirements described here for two reasons:

  1. IMAP is an important technology that provides an open standard to enable many useful features which are becoming increasingly important elements of a messaging solution. As a leading messaging company, Isode is committed to IMAP.
  2. We could not identify a suitable third party product to use. There are open source choices which are good for smaller deployments, but none of them have good enough vertical scaling and none are multi-threaded. Some commercial servers have good vertical scaling, but are weak on horizontal scaling. The open source servers that support good horizontal scaling, have poor vertical scaling and other deficiencies.