Messaging and Directory Server software used around the world in the Government, Military, Aviation and Commercial sectors.

isode.com / whitepapers /

Access, Replication and Synchronization for Mobile Email Devices

25th April 2006

Purpose

Mobile devices are increasingly tasked with handling email and associated data, such as address books and calendar information. This paper examines the many approaches to getting data onto to and off of the mobile device, looks at synchronization issues between the mobile device, desktop and server, outlines the Open Standards protocols designed to address these issues and recommends a model for mobile and desktop access to email, address book and calendars.

Mobile Device Function

A wide range of mobile devices can be used to handle email, including mobile phones, tablets PCs and laptops. Although the ideas in this paper can be applied to the larger devices, the main focus of this paper is the smaller device such as the mobile phone which we define as usually being:

• Not appropriate for primary email storage for those who save email (insufficient storage).
• Small form factor.
• Low bandwidth, high latency, and high cost network connections.

Access, Replication & Synchronization

There are a number of generic approaches to moving data to and from the mobile device. We have listed the terms used to describe those approaches below, together with an explanation how each of those terms are used within this whitepaper.

1. Access. This is where data is selectively transferred on demand, typically by a client connecting to a server and retrieving data, for example a Web browser retrieving data from a server.
2. Access with caching. When data is accessed, it may be discarded. In mobile situations, accessed data can be cached by the client as a local copy. In many situations, the result of access with caching is identical to replication.
3. Replication. Replication is where data is maintained on one system, and a copy of this data is kept up to date on another system. Replicated data flows in one direction only (from the master, to the copy).
4. Synchronization. Synchronization is where data is held on two systems, and changes may be made to that data from any data holding device. Synchronization data flows in both directions to ensure that the two copies remain the same.
5. Transfer. Where data is moved from one system to another, not directly relating to copies of data held on the two systems.
6. Remote modification. Where a client can make changes to data on a server.
   

Open Standard Protocols

There are a number of protocols that can and are being used on mobile devices to address the basic data access, replication and synchronisation concepts introduced above. All of these protocols are Internet Standards, with the exception of SyncML, a standard issued by the Open Mobile Alliance.

Web

Web (http) in its most common usage is an access protocol, where a (Web) client issues requests that result in retrieving data from the server. Web clients can also transfer data to the server, which may result in remote modification.

SyncML

SyncML is a protocol designed for generic data synchronization. It provides a framework which enables it to synchronize any type of data. A key initial target of SyncML was to synchronize data which is represented as structured records such as address books and calendars.

LDAP

LDAP (Lightweight Directory Access Protocol) provides Access (accessing data on a server using read and search) and remote modification capabilities LDAP can be used to replicate data from a server to a client, but handling of deleted data is not efficient.

CalDav

CalDav (Calendering Distributed Authoring and Versioning) is an emerging IETF specification for clients to access and update iCAL compliant calendars.

SMTP

SMTP (Simple Mail Transfer Protocol) is used to transfer messages between two systems, and is commonly used for client message submission.

POP

POP (Post Office Protocol) is an access protocol that enables an email client to retrieve messages (in the client’s inbox) from a server. The client may also delete messages on the server.
POP enables a client to access all of a message, or the first "n" lines.

IMAP

IMAP (Internet Message Access Protocol) is a sophisticated message access protocol. It goes beyond POP in two important ways:

• IMAP is aware of message structure, and can selectively access and retrieve message components (e.g., attachments).
• IMAP can support many mailboxes (folders) and so can be used for access to messages filed in shared folders as well as those in the client's inbox.

IMAP has features that enable a client to be efficiently aware of changes on the server, which means that it can be used as a replication protocol.

Functional Areas

There are a wide range of functions that a user might want access to on a mobile device. In this section we examine those functions, the approach to moving data to and from the device that these functions imply and the most efficient protocol for achieving those functions. In all cases it is likely that the user will want to access these functions from more than one device, in more than one place.

Personal Calendars

Many users maintain personal calendars and it’s natural for a user to demand access to calendar data from their mobile device when on the move and from their desktop when in the office or at home.

When they make changes to a calendar, those changes would ideally be made from the device currently in use. A typical personal calendar is of moderate size, with ongoing small incremental changes. An approach that synchronizes both calendar copies will be ideal in most situations. SyncML is the clear choice to do this, as it is the only open standard to provide the necessary functionality.

Shared Calendars

A user might also wish to access additional calendars, either shared calendars or those belonging to colleagues. Small shared calendars can be handled using the same approach as that used for personal calendars.

For a large shared calendar or for infrequent access to a colleague’s calendar, synchronization could be unduly wasteful of local storage and communication resource. A mechanism to access the calendars from the mobile device, to look at specific information, is probably the best approach. CalDav is a good optimized protocol to do this. Web access is a flexible alternate mechanism where CalDav is not supported.

Personal Address Books

Most communication components (email clients, mobile phones) will make use of one or more address books to hold contact information. While some people prefer to keep separate address books, for many users synchronization is the ideal solution where it can be supported. SyncML is the clear protocol choice to do this, as it provides optimized functionality to do this type of synchronization.

Enterprise Directory

An organization will often have a directory of users, or other shared lists of contacts used by many users. It will generally be impractical to synchronize this information with a mobile device, and so an access mechanism is preferable. LDAP is the sensible standard to use for this, as it is efficient, and can be integrated with the client address book.

Sending Email

There are different requirements for sending and receiving email , so these issues are considered separately. The core of what is needed is to transfer a message from the client to the server (message submission). SMTP is the Internet standard for mail submission and is ideal across a whole range of devices. SyncML can be used in a contrived way, Web can also be used, but it requires online access for message composition, which can be inconvenient in situations where messages need to be composed offline for submission at a later stage.

In addition to basic submission, the client may find some additional capabilities useful. The first is server side filing, for example putting a copy of the message into a “sent folder” on the server, so that sent messages can be accessed from any device.

LEMONADE is a set of new Internet Standards that extends IMAP and SMTP to better support mobile email. LEMONADE addresses the server side filing requirement by allowing a message to be created on the IMAP server, and then referenced during SMTP submission. This mechanism can also be used to manage message drafts on the server.

It might also be useful to include, in the message, information currently resident on the server. For example using attachments from messages on the server or complete messages that have been received. For large messages or attachments, optimizing message creation is an important issue. LEMONADE has a ‘forward without download’ capability, which addresses this requirement. The key to this is mechanisms in IMAP to construct messages using a mix of elements supplied by the client and data already on the server.

Client side filing might also be useful but this does not have protocol issues.

Reading Email

There are a variety of models of reading email, which lead to different protocol requirements. This section examines support of a number of different scenarios.

SCENARIO 1: Low volume email user receiving all email on the mobile device and no requirements for server archive (SMS model)
The requirement here is simply to transfer all messages from the server to the mobile device. This can be done in a number of ways: POP is a natural protocol to achieve this function. SyncML can be used, by first synchronizing the inbox, and then deleting the inbox on the server. IMAP can also be used, although most IMAP features are not needed. A key advantage over the other two options is that the IDLE command can be used to achieve immediate notification of new messages arriving. All of the following scenarios assume that the user will sometimes use a mobile device for reading email, and sometimes use a desktop system. These scenario descriptions assume general email use in home or office, and describe the mobile scenario.
SCENARIO 2: Handling all email, in a situation with relatively poor/expensive connectivity over a period of time, with a mixture of large messages
In this scenario (for example. processing email on a train journey), a key requirement is avoiding the cost of downloading the messages (especially large messages or large attachments), except when necessary. IMAP is the only suitable approach here and well optimized for this scenario as messages may be scanned and then: Deleted, or left on the server for later processing, Forwarded (without download) to another recipient, for example to get someone back in the office to handle a large document. This is a LEMONADE feature. Parts of the message selectively retrieved. Filed in an IMAP folder (without need for download).
SCENARIO 3: Retrieving email, in a short period of time, for processing offline at a later stage
In this scenario (for example, accessing email in an airport prior to a flight.), a copy of all messages is required on the mobile device. POP, IMAP and SyncML are all suitable here. IMAP can optimize this scenario, by allowing the user to select which large messages are downloaded. This could be important when bandwidth is not high, and the time with online connectivity is constrained.
SCENARIO 4: Checking for urgent emails and processing if needed
For example, checking for critical emails while traveling, and deferring other email processing to when desktop email access is available. This is best handled using IMAP using exactly the same approach as Scenario 2. The user can connect when it is convenient, process critical messages, and leave everything else for later.
SCENARIO 5: Being alerted to new messages, perhaps matching certain criteria, as soon as they arrive
IMAP is the only approach that can handle this without polling. Use of IMAP IDLE allows the client to identify new messages and alert the user to those matching the criteria. Ongoing work in the LEMONADE standardization (VFOLDER) will optimize this scenario by enabling the filtering to happen on the server.
SCENARIO 6: Accessing filed emails to retrieve specific information or a specific message
IMAP can be used to efficiently search and access filed emails. Support for this is optimized by LEMONADE, which enables efficient searching of very large folders.

Service Provision in Practice

The OMA (Open Mobile Alliance) is evaluating two options (LEMONADE and SyncML) for supporting mobile email devices. The analysis above would seem to suggest however that these two options are complementary, not competitive.

1. SyncML should be used for synchronizing personal calendars and address books, as it is ideal for these functions.
2. LEMONADE (based on IMAP/SMTP) should be used for email as SyncML does not have adequate functionality to support the same wide range of usage scenarios.

The following diagram shows how mobile and desktop access to various services would work utilising these recommended approaches to email, calendars and address books.

This picture brings all of the preceding models together. It can be seen that the primary model for personal address books and personal calendar is one of synchronization between desktop and mobile client using SyncML. This is a practical approach, supported by most mobile devices, which usually also provide the desktop part of this solution.

All of the other functionality is supported by access to servers. Good email client implementation will generally mean that some measure of caching and synchronization were appropriate will take place, particularly for the IMAP access.

Summary

This document has shown how mobile access to email and related services requires a range of protocols and mechanisms for accessing and replicating remote data. A good solution can be provided for all functionality using open standards protocols.