Messaging and Directory Server software used around the world in the Government, Military, Aviation and Commercial sectors.
Products
M-Box
- IMAP/POP Message Store.
- High performance and scalability.
- Support for IETF Mobile Device ('LEMONADE') extensions.
- Robust and stable with support for fail-over clustering & off-site hot standby.
- Comprehensive web-based administration tools.
What is M-Box?
M-Box is a high performance Internet Message Store, supporting IMAP (Internet Message Access Protocol) and POP (Post Office Protocol). M-Box uses LDAP (Lightweight Directory Access Protocol) configuration and offers good horizontal scaling. M-Box performance is examined in detail in the whitepaper "Benchmarking M-Box".
M-Box is a key element in Isode's solution for Service Providers and follows on from the IMAP development strategy we set out in this whitepaper.
M-Box is available for evaluation as part of the Isode Internet Messaging Solution for ISPs and provides the message storage and access component of an Internet Messaging solution, as illustrated below:
Email clients can access messages in M-Box using either POP or IMAP. Message switching and related services such as spam filtering are handled by M-Switch, which delivers new messages into M-Box using LMTP (Local Message Transfer Protocol). M-Box gets configuration information, including client authentication from an LDAP directory, such as M-Vault.
A more detailed example showing M-Box in the context of a full configuration is given on the ISP Solutions page. Horizontal scaling (running M-Box on multiple servers) is described later in this page.
You can evaluate all the elements of Isode's Internet Messaging Solution for ISPs (including M-Vault, M-Switch and M-Vault) by visiting this page.
Key Benefits
M-Box has many functions, which are described below. Reasons why this product may be of particular interest include:
- High performance. M-Box has an optimized database design and multi-threaded architecture, which gives high performance and allows a very large number of users to be supported on a single server.
- Horizontal scaling. Multiple M-Box can robustly share a single file store, enabling straightforward service scaling for very large deployments.
- Simple database format. M-Box uses a simple format mapping directly to the underlying file store, with one message per file and one directory entry per account. This gives natural robustness, and simplifies account provisioning.
- LDAP directory based configuration. M-Box stores configuration and authentication information in the Directory, which allows sharing between multiple instances of M-Box and M-Switch, as well as enabling straightforward integration with a provisioning system.
- Web based management. The Internet Messaging Administrator allows email administrators to create and manage user email accounts and shared folders, edit the message store configuration (including message size quotas and delivery options) and examine the current directory configuration (more).
- Mobile device support. M-Box is designed to support mobile email devices, and in particular has support for some of the LEMONADE specifications.
- Rock Solid. M-Box has exceptional robustness and stability, including support for fail-over clustering and Off Site Hot Standby (Disaster Recovery).
- Online backup can be performed using standard backup tools, as a consequence of the M-Box database architecture. This enables straightforward disaster recovery.
- Security. There are a variety of security features, including SASL for POP and IMAP access.
- Audit and event logging will be added in a future release, using the same mechanisms as other Isode products. The current release uses syslog for critical events.
M-Box Architecture
An M-Box server is based around a message database, and has three multi-threaded servers that access this database:
- imapd: supports access using IMAP.
- pop3d: supports access using POP.
- lmptd: supports message delivery using LMTP.
The most important part of the M-Box architecture is the database design. M-Box uses a database abstraction layer, so that it is possible to use different database back-ends. This has been prototyped with several back-ends, although a single database back-end is currently being provided. Future releases of M-Box will take advantage of this architectural flexibility.
The underlying database is mapped onto the underlying file store using a format which we call “mdir”. This uses the basic approach of the popular maildir format, which enables mailbox sharing without use of file system locks. Mdir differs from maildir primarily to enable performance improvements, particularly in support of IMAP. Key features of the mdir format:
- No locks. This is critical for high performance, and robust horizontal scaling.
- Each message in a single file. This simple approach gives inherent high robustness, as any data corruption will be constrained to a single message. It also allows standard file system backup mechanisms to be used for online backup.
- Use of file store hierarchy. Folders and mailboxes are structured using the file store hierarchy. This approach is inherently robust and allows use of standard OS tools on the database. A key benefit is that each account is represented by a single directory hierarchy, which simplifies provisioning support and in particular account deletion.
A quick analysis might suggest that this simple approach, while very robust, would not give good performance. In fact, exactly the opposite is true. Modern files systems are highly optimized for exactly the sort of operations that are performed by M-Box. In particular:
- File system directories are indexed using b-trees, and have optimized layout on disk. Use of large directories with key indexing information in the filename is therefore very efficient.
- Files are intelligently laid out on the disk, which means that use of relatively small block size (good to gain high disk utilization, as messages are generally small) does not impact performance for large files.
In essence, M-Box is working with the tuning of the underlying file system, which is one of the reasons for its excellent performance. Our tests have shown that M-Box is faster than servers that use complex approaches to optimize performance, and that the approach taken gives benefits of simplicity, robustness and high performance.
A key benefit that mdir provides is support of horizontal scaling as illustrated above. This architecture allows multiple M-Box servers to access a single file store. This architecture gives “horizontal scaling”, and enables additional M-Box servers to be added to support more users. Load balancing between the M-Box servers will usually be achieved by use of a network load balancer. The file store will be chosen with appropriate size, performance and expansion options for the deployment. It may be a file system on one of the M-Box servers, but in most cases it is likely to be a separate network file server appliance. The file access protocol will depend on performance and scaling targets for the total system. Options include:
- NFS (Network File System). The traditional protocol for this sort of functionality.
- iSCSI. A mapping of the SCSI (Small Computer Systems Interface) protocol commonly used onto IP. iSCSI can be used with a clustering file system to provide a low cost, higher performance alternative to NFS.
- SAN (Storage Area Network). A more expensive, higher performance option.
Although this approach gives an elegant way to scale an M-Box deployment by adding more servers, the major advantage of this architecture is in simplified management. This architecture leads to user mailboxes being deployed on a single file store, with each account having a separate directory. Increase in storage is simply a question of increasing the size of the file server. This makes it easy to add and remove accounts.
Many mailbox servers, particularly those with complex databases, lock mailboxes to specific servers. This leads to two problems:
- In order to make mailbox location transparent to the end user, a tier of application servers will generally be introduced, so that the user gets redirected to the right server.
- When servers are added, or usage grows, there will be the requirement to migrate users between servers. This adds product and operational complexity.
M-Box's architecture avoids these problems.
Performance
M-Box is fast. Our own benchmarking indicates that M-Box is significantly faster than open source implementations and commercial competitors. Whilst license restrictions prevent us from publishing comparison figures against commercial competitors, we'd encourage you to perform your own tests and will support you in those tests.
While there is much detail to consider, there are three basic questions to answer with regards to performance:
- How fast is the database? This is best tested by a single connection performing operations on a mailbox. M-Box performs much better than most, and as well as the best (Cyrus).
- What changes when you add multiple connections? M-Box's multi-threaded architecture means that it performs significantly better than the other stores.
- Does it scale horizontally? The other stores that offer horizontal scaling have very bad answers to Q1 and Q2.
IMAP and POP Protocol Support
IMAP and POP are both protocols with a large number of optional extensions. M-Box has comprehensive support of these optional extensions, making it ideal for environments supporting a wide mix of email clients. Details of protocol support are listed in the Internet Mail Conformance section below.
M-Box is designed to support mobile devices, and conformance to some of the new LEMONADE specifications is included, prior to final standardization of this work. The IETF's LEMONADE (License to Enhanced Mobile Oriented And Diverse Endpoints) working group is addressing messaging standard extentions for optimized open standard mobile messaging. You can read more about LEMONADE and Mobile Messaging in our whitepaper, Open Standards: The best approach for mobile messaging.
SIEVE, which is a delivery time scripting language for email processing is supported by lmtpd. SIEVE enables users to manage actions such as automatic email filing, and vacation alerts. Personal sieve scripts are configured in the directory and may be managed by IMA. There is also a "global" sieve script, so that custom processing can be defined for all users.
Configuration Management
The configuration management of M-Box is integrated with the directory based configuration of the messaging infrastructure. M-Box configuration information is held in the (LDAP) directory, which enables client/server management of data and sharing of information between M-Box servers. This includes configuration information for the M-Box server, but the most important information is on users.
The core model is "one user; one account; one directory entry". While this can be managed with Isode’s directory data entry tools, in many deployments account management and user option selection will be integrated with the customer provisioning system, with the directory forming a back end to this.
More details, including the LDAP attributes used to configure M-Box, can be found in the M-Box Administration Manual, available for download from this site.
M-Box server configuration is done using the Web based Internet Messaging Administration (IMA) applications.
|
Account Administration
Web-based administration if provided by the Internet Messaging Administration application.
Email administrators can use the IMA to:
- Create and manage user email accounts
- Create and manage shared email folders
- Create and manage groups of users (used in access control lists and for shared folders)
- Edit the message store configuration (including message size quotas and delivery options)
- Examine the current directory configuration.
You can find out more about the Internet Messaging Administrator by clicking here.
Operational Management
M-Box specific operational control is provided by the msadm tool, which enables checking of accounts and msstat which provides connection information. Msadm provides scripted creation of accounts, and enables message expiration, so that old message can be removed. M-Box provides audit logging of key events, and uses Isode events for errors and operational information. This is described in the page on Event Handling and Audit Logging.
M-Box provides SNMP monitoring of general status and network connection. Events may also be monitored by SNMP, with alerts using SNMP Notifications (TRAPs). Further information is given on the page Isode's SNMP Architecture.
Migration
M-Box provides a fully automated, on-the-fly migration system, capable of migrating both message data and authentication information from an existing POP or IMAP service.
In a one-shot variant of the Gateway model, the single POP mailbox, or all the user's personal IMAP mailboxes, are migrated dynamically on login, using only standard, secured POP3 or IMAP connections. On completion, the user is automatically provisioned with an account in the directory.
In the majority of cases, this allows for a switchover from an existing system entirely transparent to the user, with no pre-poulation of account data required.
This reduces both the downtime and administrative load dramatically, allowing rollout to be achieved with zero-downtime.
Authentication & Security
Authentication of IMAP and POP users may be performed using SASL, built over Isode's SASL infrastructure. Data confidentiality may be provided either by SASL and/or by TLS.
Internet Mail Conformance
General
| RFC 1123 | Requirements for Internet hosts - application and support, R. Braden, October 1989 |
|---|
| RFC 2045 | Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies, N. Freed, N. Borenstein, November 1996 |
|---|
| RFC 2046 | Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types, N. Freed, N. Borenstein, November 1996 |
|---|
| RFC 2047 | (Multipurpose Internet Mail Extensions) MIME Part Three: Header Extensions for Non-ASCII Text, K. Moore, November 1996 |
|---|
| RFC 2049 | (Multipurpose Internet Mail Extensions) MIME Part Five: Conformance Criteria and Examples, N. Freed, N. Borenstein, November 1996 |
|---|
| RFC 3462 | The Multipart/Report Content Type for the Reporting of Mail System Administrative Messages, G. Vaudreuil, January 2003 |
|---|
IMAP
| RFC 2087 | IMAP4 QUOTA extension, J. Myers, January 1997 |
|---|
| RFC 2088 | IMAP4 non-synchronizing literals, J. Myers, January 1997 |
|---|
| RFC 2177 | IMAP4 IDLE command, B. Leiba, June 1997 |
|---|
| RFC 2342 | IMAP4 Namespace, M. Gahrns & C. Newman, May 1998 |
|---|
| RFC 3501 | INTERNET MESSAGE ACCESS PROTOCOL - Version 4rev1, M. Crispen, March 2003 |
|---|
| RFC 3516 | IMAP4 Binary Content Extension, L. Nerenberg, April 2003 |
|---|
| RFC 3691 | Internet Message Access Protocol (IMAP) UNSELECT command, A. Melnikov, February 2004 |
|---|
| RFC 4314 | IMAP4 ACL extension, A. Melnikov, December 2005 |
|---|
| RFC 4315 | INTERNET MESSAGE ACCESS PROTOCOL - UIDPLUS extension, M Crispen, December 2005 |
|---|
| RFC 4467 | Internet Message Access Protocol (IMAP) - URLAUTH extension, M. Crispin, May 2006 |
|---|
| RFC 4469 | Internet Message Access Protocol (IMAP) - CATENATE extension, P. Resnick, April 2006 |
|---|
| Internet Email to Support Diverse Service Environments (Lemonade) Profile, S. Meas, A. Melnikov, June 2006 |
| RFC 4551 | IMAP Extenstion for Conditional STORE Operation of Quick Flag Changes Resyncronization, A. Melnikov, S.Hole, June 2006 |
|---|
| IMAP4 extension to SEARCH command for controlling what kind of information is returned, A. Melnikov, D. Cridland, November 2006 |
| The IMAP Extension for SASL Initial Client Response, Robert Siemborski, A. Gulbrandsen Sept 2007 |
| The IMAP COMPRESS Extension, Arnt Gulbandsen, August 2007 |
| WITHIN Search extensions to the IMAP Protocol, Eric Burger, September 2007 |
| IMAP URL Scheme, A. Melnikov, C Newman, November 2007 |
| The IMAP ENABLE Extension, A. Melnikov, A. Gulbrandsen, March 2008 |
| Contexts for IMAP4, D. Cridland, C. King, June 2007 |
| INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS, M. Crispin, K. Murchison, May 2004 |
| IMAP4 LIST Command extensions, B. Leiba, A. Melnikov, September 2006 |
| IMAP4 Extensions for Quick Mailbox Resynchronization, A. Melnikov, D. Cridland, C. Wilson, June 2007 |
LMTP
| RFC 1652 | SMTP Service Extension for 8-bit MIME transport, J. Klensin, N. Freed, M. Rose, E. Stefferud, D. Crocker, July 1994 |
|---|
| RFC 2033 | Local Mail Transfer Protocol, J. Meyers, October 1996 |
|---|
| RFC 2034 | SMTP Service Extension for Returning Enhanced Error Codes, N Freed, October1996 |
|---|
| RFC 2920 | SMTP Service Extension for Command Pipelining, N. Freed, September 2000 |
|---|
| RFC 3848 | ESMTP and LMTP Transmission Types Registration, C. Newman, July 2004 |
|---|
SIEVE
| Sieve: A Mail Filtering Language, T. Showalter, January 2008 |
| Sieve Email Filtering: Relational Extension, W. Segmuller, B. Leiba January 2008 |
| Sieve Email Filtering -- Subaddress Extension, K. Murchison, January 2008 |
| Sieve Extension: Copying Without Side Effects, J. Degener, October 2004 |
| Sieve Email Filtering: Vacation Extension, T. Showalter, N. Freed, January 2008 |
| A Protocol for Remotely Managing Sieve Scripts, T. Martin, A. Melnikov, January 2008 |
POP
| RFC 1734 | POP3 AUTHentication command, J. Myers, December 1994 |
|---|
| RFC 1939 | Post Office Protocol - Version 3, J. Myers, M. Rose, May 1996 |
|---|
| RFC 2449 | POP3 Extension Mechanism, R. Gellens, C. Newman, L. Lundblade, November 1998 |
|---|
| RFC 2595 | Using TLS with IMAP, POP3 and ACAP, C. Newman, June 1999 |
|---|
Availability
M-Box is available on Linux, Solaris and Windows. Details on supported platforms and versions can be found here.
