M-Box POP/IMAP Server
M-Box is a high performance Internet Message Store, supporting IMAP
(Internet Message Access Protocol) and POP (Post Office Protocol). M-Box
uses LDAP (Lightweight Directory Access Protocol) configuration and
offers good horizontal scaling. M-Box is a key element in Isode's Mailbox Services and provides the message storage
and access component of an Internet Messaging solution, as illustrated
Email clients can access messages in M-Box using either POP or IMAP.
Message switching and related services such as spam filtering are handled
by M-Switch, which delivers new messages
into M-Box using LMTP (Local Message Transfer Protocol). M-Box gets
configuration information, including client authentication from an LDAP
directory, such as M-Vault.
A more detailed example showing M-Box in the context of a full configuration
is given on the Mailbox Services page.
Horizontal scaling (running M-Box on multiple servers) is described
later in this page. You can evaluate all the elements of Isode's Internet Messaging Solution (including M-Vault, M-Switch and M-Vault).
M-Box has many functions, which are described below. Reasons why this
product may be of particular interest include:
- High performance. M-Box has an optimized database design and multi-threaded
architecture, which gives high performance and allows a very large
number of users to be supported on a single server.
- Horizontal scaling. Multiple M-Box can robustly share a single
file store, enabling straightforward service scaling for very large
- Simple database format. M-Box uses a simple format mapping directly
to the underlying file store, with one message per file and one directory
entry per account. This gives natural robustness, and simplifies account
- LDAP directory based configuration. M-Box stores configuration
and authentication information in the Directory, which allows sharing
between multiple instances of M-Box and M-Switch, as well as enabling
straightforward integration with a provisioning system.
- Web based management. The Internet Messaging Administrator allows
email administrators to create and manage user email accounts and
shared folders, edit the message store configuration (including message
size quotas and delivery options) and examine the current directory
- Mobile device support. M-Box is designed to support mobile email
devices, and in particular has support for some of the LEMONADE specifications.
- Rock Solid. M-Box has exceptional robustness and stability, including
support for fail-over clustering and Off Site Hot Standby (Disaster Recovery).
- Online backup can be performed using standard backup tools, as
a consequence of the M-Box database architecture. This enables straightforward
- Security. There are a variety of security features, including SASL
for POP and IMAP access.
- Audit and event logging will be added in a future release, using
the same mechanisms as other Isode products. The current release uses
syslog for critical events.
- Flexible message processing on delivery, controlled by SIEVE.
An M-Box server is based around a message database, and has three multi-threaded
servers that access this database:
- imapd: supports access using IMAP.
- pop3d: supports access using POP.
- lmptd: supports message delivery using LMTP.
The most important part of the M-Box architecture is the database design.
M-Box uses a database abstraction layer, so that it is possible to use
different database back-ends. This has been prototyped with several
back-ends, although a single database back-end is currently being provided.
Future releases of M-Box will take advantage of this architectural flexibility.
The underlying database is mapped onto the underlying file store using
a format which we call “mdir”. This uses the basic approach
of the popular maildir format, which enables mailbox sharing without
use of file system locks. Mdir differs from maildir primarily to enable
performance improvements, particularly in support of IMAP. Key features
of the mdir format:
- No locks. This is critical for high performance, and robust horizontal
- Each message in a single file. This simple approach gives inherent
high robustness, as any data corruption will be constrained to a single
message. It also allows standard file system backup mechanisms to
be used for online backup.
- Use of file store hierarchy. Folders and mailboxes are structured
using the file store hierarchy. This approach is inherently robust
and allows use of standard OS tools on the database. A key benefit
is that each account is represented by a single directory hierarchy,
which simplifies provisioning support and in particular account deletion.
A quick analysis might suggest that this simple approach, while very
robust, would not give good performance. In fact, exactly the opposite
is true. Modern files systems are highly optimized for exactly the sort
of operations that are performed by M-Box. In particular:
- File system directories are indexed using b-trees, and have optimized
layout on disk. Use of large directories with key indexing information
in the filename is therefore very efficient.
- Files are intelligently laid out on the disk, which means that
use of relatively small block size (good to gain high disk utilization,
as messages are generally small) does not impact performance for large
In essence, M-Box is working with the tuning of the underlying file
system, which is one of the reasons for its excellent performance. Our
tests have shown that M-Box is faster than servers that use complex
approaches to optimize performance, and that the approach taken gives
benefits of simplicity, robustness and high performance.
A key benefit that mdir provides is support of horizontal scaling as
illustrated above. This architecture allows multiple M-Box servers to
access a single file store. This architecture gives 'horizontal
scaling', and enables additional M-Box servers to be added to
support more users. Load balancing between the M-Box servers will usually
be achieved by use of a network load balancer. The file store will be
chosen with appropriate size, performance and expansion options for
the deployment. It may be a file system on one of the M-Box servers,
but in most cases it is likely to be a separate network file server
appliance. The file access protocol will depend on performance and scaling
targets for the total system. Options include:
- NFS (Network File System). The traditional protocol for this sort
- iSCSI. A mapping of the SCSI (Small Computer Systems Interface)
protocol commonly used onto IP. iSCSI can be used with a clustering
file system to provide a low cost, higher performance alternative
- SAN (Storage Area Network). A more expensive, higher performance
Although this approach gives an elegant way to scale an M-Box deployment
by adding more servers, the major advantage of this architecture is
in simplified management. This architecture leads to user mailboxes
being deployed on a single file store, with each account having a separate
directory. Increase in storage is simply a question of increasing the
size of the file server. This makes it easy to add and remove accounts.
Many mailbox servers, particularly those with complex databases, lock
mailboxes to specific servers. This leads to two problems:
- In order to make mailbox location transparent to the end user, a
tier of application servers will generally be introduced, so that
the user gets redirected to the right server.
- When servers are added, or usage grows, there will be the requirement
to migrate users between servers. This adds product and operational
M-Box's architecture avoids these problems.
M-Box is fast. Our own benchmarking indicates
that M-Box is significantly faster than open source implementations
and commercial competitors. Whilst license restrictions prevent us from
publishing comparison figures against commercial competitors, we'd encourage
you to perform your own tests and will support you in those tests.
While there is much detail to consider, there are three basic questions
to answer with regards to performance:
- How fast is the database? This is best tested by a single connection
performing operations on a mailbox. M-Box performs much better than
most, and as well as the best (Cyrus).
- What changes when you add multiple connections? M-Box's multi-threaded
architecture means that it performs significantly better than the
- Does it scale horizontally? The other stores that offer horizontal
scaling have very bad answers to Q1 and Q2.
IMAP and POP Protocol Support
IMAP and POP are both protocols with a large number of optional extensions.
M-Box has comprehensive support of these optional extensions, making
it ideal for environments supporting a wide mix of email clients. Details
of protocol support are listed in the Internet
Mail Conformance section below.
M-Box is designed to support mobile devices, and conformance to some
of the new LEMONADE specifications is included, prior to final standardization
of this work. The IETF's LEMONADE (License to Enhanced Mobile Oriented And Diverse Endpoints) working group is
addressing messaging standard extensions for optimized open standard
mobile messaging. You can read more about LEMONADE and Mobile Messaging
in our whitepaper, [Open
Standards: The best approach for mobile messaging].
SIEVE, which is a delivery time scripting language for email processing
is supported by lmtpd. SIEVE enables users to manage actions such as
automatic email filing, and vacation alerts. Personal sieve scripts
are configured in the directory and may be managed by IMA. There is
also a "global" sieve script, so that custom processing can
be defined for all users.
The configuration management of M-Box is integrated with the directory
based configuration of the messaging infrastructure. M-Box configuration
information is held in the (LDAP) directory, which enables client/server
management of data and sharing of information between M-Box servers.
This includes configuration information for the M-Box server, but the
most important information is on users.
The core model is "one user; one account; one directory entry".
While this can be managed with Isode’s directory data entry tools,
in many deployments account management and user option selection will
be integrated with the customer provisioning system, with the directory
forming a back end to this.
More details, including the LDAP attributes used to configure M-Box,
can be found in the M-Box Administration
Manual, available for download from this site. M-Box server configuration is done using the Web based Internet
Messaging Administration (IMA) applications.
Web-based administration if provided by the Internet Messaging Administration
Email administrators can use the IMA to:
- Create and manage user email accounts
- Create and manage shared email folders
- Create and manage groups of users (used in access control lists
and for shared folders)
- Edit the message store configuration (including message size quotas
and delivery options)
- Examine the current directory configuration.
You can find out more about the Internet Messaging Administrator by clicking here.
M-Box specific operational control is provided by the msadm tool, which
enables checking of accounts and msstat which provides connection information.
Msadm provides scripted creation of accounts, and enables message expiration,
so that old message can be removed. M-Box provides audit logging of
key events, and uses Isode events for errors and operational information.
This is described in the page on Event
Handling and Audit Logging.
M-Box provides SNMP monitoring of general status and network connection.
Events may also be monitored by SNMP, with alerts using SNMP Notifications
(TRAPs). Further information is given on the page Isode's
M-Box provides a fully automated, on-the-fly migration system, capable
of migrating both message data and authentication information from an
existing POP or IMAP service.
In a one-shot variant of the Gateway model,
the single POP mailbox, or all the user's personal IMAP mailboxes, are
migrated dynamically on login, using only standard, secured POP3 or
IMAP connections. On completion, the user is automatically provisioned
with an account in the directory.
In the majority of cases, this allows for a switchover from an existing
system entirely transparent to the user, with no pre-population of account
This reduces both the downtime and administrative load dramatically,
allowing rollout to be achieved with zero-downtime.
Authentication & Security
Authentication of IMAP and POP users may be performed using SASL, built
over Isode's SASL infrastructure. Data confidentiality
may be provided either by SASL and/or by TLS.
M-Box supports IMAP Access Control, which can be managed by IMAP clients,
for example to enable folder sharing,
||Requirements for Internet hosts - application and support. R. Braden, October 1989
||POP3 Authentication command. J. Myers, December 1994
||Post Office Protocol - Version 3. J. Myers, M. Rose, May 1996
||Local Mail Transfer Protocol. J. Meyers, October 1996
||SMTP Service Extension for Returning Enhanced Error Codes. N. Freed, October1996
||Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies. N. Freed, N. Borenstein, November 1996
||Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types. N. Freed, N. Borenstein, November 1996
||Multipurpose Internet Mail Extensions (MIME) Part Three: Header Extensions for Non-ASCII Text. K. Moore, November 1996
||Multipurpose Internet Mail Extensions (MIME) Part Five: Conformance Criteria and Examples. N. Freed, N. Borenstein, November 1996
||IMAP4 QUOTA extension. J. Myers, January 1997
||IMAP4 non-synchronizing literals. J. Myers, January 1997
||IMAP4 IDLE command, B. Leiba, June 1997
||IMAP4 Namespace. M. Gahrns & C. Newman, May 1998
||POP3 Extension Mechanism. R. Gellens, C. Newman, L. Lundblade, November 1998
||Using TLS with IMAP, POP3 and ACAP. C. Newman, June 1999
||SMTP Service Extension for Command Pipelining. N. Freed, September 2000
||The Multipart/Report Content Type for the Reporting of Mail System
Administrative Messages. G. Vaudreuil, January 2003
||Internet Message Access Protocol - Version 4rev1. M. Crispen, March 2003
||IMAP4 Binary Content Extension, L. Nerenberg, April 2003
||Internet Message Access Protocol (IMAP) UNSELECT command. A. Melnikov, February 2004
||Sieve Extension: Copying Without Side Effects. J. Degener, October 2004
||ESMTP and LMTP Transmission Types Registration. C. Newman, July 2004
||IMAP4 ACL extension. A. Melnikov, December 2005
||INTERNET MESSAGE ACCESS PROTOCOL - UIDPLUS extension, M Crispen, December
||Internet Message Access Protocol (IMAP) - URLAUTH extension. M. Crispin, May 2006
||Internet Message Access Protocol (IMAP) - CATENATE extension. P. Resnick, April 2006
||Internet Email to Support Diverse Service Environments (Lemonade) Profile. S. Meas, A. Melnikov, June 2006
||IMAP Extenstion for Conditional STORE Operation of Quick Flag Changes
Resyncronization. A. Melnikov, S.Hole, June 2006
||IMAP4 extension to SEARCH command for controlling what kind of information is returned, A. Melnikov, D. Cridland, November 2006
||The IMAP Extension for SASL Initial Client Response. Robert Siemborski, A. Gulbrandsen Sept 2007
||The IMAP COMPRESS Extension. Arnt Gulbandsen, August 2007
||WITHIN Search extensions to the IMAP Protocol. Eric Burger, September 2007
||IMAP URL Scheme. A. Melnikov, C Newman, November 2007
||The IMAP ENABLE Extension. A. Melnikov, A. Gulbrandsen, March 2008
||IMAP4 Extensions for Quick Mailbox Resynchronization, A. Melnikov, D. Cridland, C. Wilson, March 2008
||Sieve: A Mail Filtering Language. T. Showalter, January 2008
||Sieve Email Filtering: Vacation Extension. T. Showalter, N. Freed, January
||Sieve Email Filtering: Relational Extension. W. Segmuller, B. Leiba January
||Sieve Email Filtering: Subaddress Extension. K. Murchison, January 2008
||Internet Message Access Protocol - SORT and THREAD Extensions, M. Crispin, K. Murchison, June 2008
||Contexts for IMAP4, D. Cridland, C. King, July 2008
||IMAP4 LIST Command extensions, B. Leiba, A. Melnikov, June 2008
||Sieve Email Filtering: Date and Index Extensions. N. Freed, July 2008
||A Protocol for Remotely Managing Sieve Scripts. T. Martin, A. Melnikov,
||SMTP Service Extension for 8-bit MIME transport, J. Klensin, N. Freed, M. Rose, E. Stefferud, D. Crocker, March 2011
||IMAP LIST Extension for Special-Use Mailboxes, B. Leiba, J. Nicolson, March 2011
||Internet Message Access Protocol (IMAP) - MOVE Extension, A. Gulbrandsen, N. Freed, Ed. Oracle, January 2013
|| The IMAP SEARCH=INTHREAD and THREAD=REFS Extensions, Arnt Gulbrandsen, July 2010