Messaging and Directory Server software used around the world in the Government, Military, Aviation and Commercial sectors.
Products
M-Switch SMTP
M-Switch
SMTP is a high performance, highly flexible and robust Internet Message
Switch using directory configuration. It can be used standalone, or
as the base switching system for M-Switch Anti-Spam.
Deployment Targets
M-Switch has two basic deployment targets described below: Boundary Message Switching and ISP.
Boundary Messaging
The boundary configuration shown above is where M-Switch SMTP performs an SMTP (Simple Mail Transfer Protocol) message switching. This boundary messaging configuration will often be used in conjunction with a firewall, to control inbound and outbound messages, and to provide anti-spam and anti virus capabilities described in more detail under M-Switch Anti-Spam.
This boundary switch may route internally to multiple departmental systems, and may do address rewriting to provide a uniform external appearance. M-Switch can use LDAP to access multiple departmental directories to perform boundary address validation.
ISP
M-Switch is the message switching component of Isode’s ISP solution. Details of the whole solution are given on the ISP Solutions page.
Key Benefits of M-Switch SMTP
The M-Switch has many functions, which are described below. Reasons why this product may be of particular interest include:
Flexibility
The architecture of the Message Switch, the management tools, and directory based configuration combine to give a very high degree of customer flexibility.
Performance
M-Switch SMTP has very high throughput and low latency.
Excellent scheduling and operational characteristics
The Queue Manager (QMGR) and channel architecture described below enables a sophisticated scheduling approach, which combined with the Message Switch's queue structure leads to a product which works exceedingly well in demanding operational environments. More details are given here.
Rock Solid
M-Switch has exceptional robustness and stability, including support for fail-over clustering and Off Site Hot Standby (Disaster Recovery).
Mobile Messaging Support
M-Switch SMTP supports the Lemonade profile.
LDAP directory based configuration
The Message Switch uses LDAP as its preferred configuration mechanism,
which enables sharing of routing and configuration information, and
flexible client/server management. Mailbox information is also held
in LDAP directories, which can be integrated with an ISP provisioning
system, or make use of existing (departmental) LDAP directories.
|
Management Features
The product has a wide range of management features, including configuration, SNMP monitoring, distribution lists, content conversion, message tracking, statistics, quarantine management and address mapping control. For more information see the Message Operator Interface page.
Address Handling
There are sophisticated features to deal with complex email addressing situations, including alias and redirect capabilities and address mapping mechanisms.
Security
There are a variety of security features, including SASL verified submission, TLS data confidentiality, audit database, and a general purpose message authorization control.
Managing M-Switch SMTP
M-Switch SMTP has a number of leading edge management features which include:
- Directory (LDAP) based configuration.
- Flexible address mapping and manipulation.
- Message authorization and access control.
- Audit database, statistics and tracking.
Managing M-Switch SMTP is described separately from this page in two places:
- The tools for monitoring and controlling an operational system are described on the Messaging Operational Management and include audit database, message tracking and message archiving.
- Configuration of M-Switch SMTP is described on the Messaging Configuration Management page.
- For an ISP configuration, user information is held in an LDAP directory and this information is shared by M-Box and M-Switch. User management is handled by the Internet Messaging Administrator.
The rest of this page looks at the core M-Switch SMTP functionality.
Message Switch Functionality
|
Architecture
The overall M-Switch SMTP architecture and the primary processes that comprise M-Switch SMTP are illustrated in the diagram to the right (click on the thumbnail for the full version), and the following sections summarize how these components work and interact. The event handling system is described later, but not illustrated.
Message Queue
The (on disk) message queue is the centre of M-Switch. Arriving messages are written to the disk and secured, which is central to reliable store and forward messaging. Messages may then be checked and processed locally, before being sent on and removed from the disk. Typically, messages will remain in the queue for a very short while, often less than a second.
Channels
Most M-Switch processes are channels, shown in purple on the diagram above. The exceptions are QMGR (which is special) and three processes associated with the audit database. Channels perform many functions, including bringing messages into M-Switch (e.g., SMTP Server), delivering or transferring messages out of M-Switch, and processing messages within M-Switch.
Queue Manager (QMGR)
QMGR is the operational core of M-Switch, communicating with channels and with management interfaces. Key QMGR tasks:
- Starting and stopping channel processes.
- Scheduling active processing and delivery of messages.
- Controlling number of connections to the SMTP server.
- Receiving status information from channels, on connections and message transfer status.
- Responding to information and control requests from management processes.
To achieve this, QMGR makes use of three protocols:
- An internal channel communication protocol.
- AgentX to enable SNMP monitoring (not included in R12.0)
- SOM (Switch Operational Management Protocol), which is used by Isode’s management tools and provides strong authentication and data confidentiality using TLS and SASL. SOM Client Libraries are available for use by third party management tools (example here).
QMGR is a multi-threaded event driven process that holds an internal representation of the status of all queued messages, channels and connections. By handling all of this with a single process that does not need to look at the queues on disk, very sophisticated scheduling, control and status reporting is available. QMGR is designed to efficiently manage very large message queues (hundreds of thousands of messages). All information held by QMGR and control options are available through the MConsole tool, described here.
SMTP & LMTP
Isode provides full support for Simple Mail Transfer Protocol (SMTP) as specified in RFC 2821, conforming to the Internet host requirements for messaging (RFC 1123) including message submission (RFC 4409). SMTP support includes a number of auxiliary Internet Standard related to general capabilities and Lemonade support. A full list is given in the conformance section at the end of this document.
LMTP (RFC 2033) is a variant of SMTP optimized for delivery to local message stores, also supported by Isode’s M-Box IMAP/POP server.
Message Transfer by SMTP out of M-Switch is provided by the SMTP channel and Message Delivery by LMTP is provided by the LMTP channel. M-Switch SMTP may run many instances of each of these channels.
Inbound SMTP is handled by a single multi-threaded SMTP Server process (channel), so that new SMTP connections do not require new processes.
Message submission uses the SMTP protocol. The Internet Standard for Message Submission (RFC 4409) gives a special port for this (587), distinct from the SMTP port used for message transfer (25). Message submission is frequently authenticated, whereas this is unusual for message transfer. The SMTP Server can be configured to listen on both ports, or two SMTP servers can be used (one for each port).
Message Submission Library
The Message Submission Library provides a standard internal service for putting messages onto the message queue, which ensures that per message functions such as authentication, routing, and processing calculations are applied consistently and correctly. In older versions of M-Switch, this functionality was provided in a separate process. Key functions provided by the submission library are:
- Sender validation.
- Recipient validation and message routing.
- Determining what internal processing is needed.
- Policy and authorization checks.
- Writing the message to disk.
The Message Submission Library is used in many components of M-Switch, and in particular in the SMTP Server.
M-Switch Configuration
M-Switch SMTP generally uses directory configuration, which has many advantages. Table based configuration is also provided, as it is useful for some specialized deployments.
The M-Switch GUI for managing directory configuration is EMMA, which is described in the Messaging Configuration Management page.
M-Switch configuration is used by all of the M-Switch processes. Logically, configuration information from the directory is used by all processes. As changes to configuration are generally infrequent, this is implemented by QMGR obtaining configuration information from the directory using LDAP, checking for updates, and sharing it with the other processes.
Self Test
In order to optimize performance, the number of channels running, number of each channel type running, number of connections to a single MTA, will vary dependent on many factors, including remote system, network and local hardware performance. Manual optimization is complex, and in any event the optimal configuration is likely to vary over time. For this reason, M-Switch does a sophisticated “self test”, to measure performance and adapts configuration accordingly. This enables M-Switch to adapt its operational parameters to different hardware and network environments.
Message Routing
Routing of messages to other SMTP servers uses the domain part of the email address and is generally handled using the DNS (Domain Name System). M-Switch also allows for connections to be configured using EMMA to specific SMTP hosts. Reasons for doing this include:
- Internal routing that over-rides DNS based routing.
- Configuring security options, such as TLS for data confidentiality and SASL for peer authentication using passwords or strong authentication.
- Configuring one or more permanent SMTP connections, which will reduce message switching latency by removing the time needed to establish a connection.
M-Switch also routes based on the whole email address. This is referred to as LASER routing, after the Internet working group that developed a specification with this name that did not get published as a standard. M-Switch broadly follows that specification, with a number of extensions, and options to deal with directory servers that store information in a different way.
The key concept in LASER routing is that you search in an LDAP directory for a mailbox of the form joe.user@mycorp.com. Routing succeeds when an entry is correctly matched. Additional attributes in the entry can control with channel and host the message is then sent to. Typically, the match will use a default LMTP connection to a local M-Box server.
Three basic ways that LASER routing may be used in conjunction with domain based routing:
- Single domain system, such as a consumer ISP or enterprise. If the domain is local, LASER routing is used, otherwise domain routing is used.
- Hosted service, handling multiple domains on one server. LASER routing is used to a single directory server. If there is a match, the message is handled according to the match. If not, domain routing is used for authorized message submission.
- Boundary switching to multiple departmental servers. Domain routing looks to identify locally configured domains. A match will identify the departmental LDAP server against which to do the LASER lookup, and the SMTP server to which matched addresses should be routed. This approach allows the boundary switch to fully validate email addresses by making use of existing LDAP servers.
M-Switch has flexible configuration that can support these setups and others. In addition to the core LASER routing, M-Switch also provides:
- A mechanism to specify alternate mailbox addresses.
- A list channel, that expands managed distribution lists.
- Sub-addressing support (joe+XXX@myorg.com).
Event Handling
M-Switch uses the Isode event handling system. Multiple event streams can be configured, with event access using Isode’s Event View, Unix Syslog, or Windows Event Manager. Isode events are documented with description and severity.
More details on the event system are given in this whitepaper on operational monitoring.
Audit Logging & Audit Database
Detailed information on messages handled and processing status are recorded in an audit log. This audit log is processed by the Message Audit Service which is stored in an audit database. The audit database may be local or remote and used by one or more M-Switch servers.
The Audit Database is accessed by MConsole and the Message Operator Interface (MOI) which use this information to provide:
- Statistics and reports on M-Switch usage and performance.
- Message Tracking.
- Message Archive Retrieval.
- Quarantine management.
Further details are given in Message Operator Interface Overview.
Anti-Spam and Anti-Virus Channels
Detailed information on anti-spam and anti-virus are given under M-Switch Anti-Spam and M-Switch Anti-Virus. This section summarizes how these capabilities are provided in M-Switch. A number of anti-spam measures are implemented in the SMTP Server. Although messages may be rejected at the SMTP level, the SMTP server usually simply records the results of anti-spam checks in the message, so that they can be used in conjunction with other checks done in the anti-spam channel.
The core anti-spam function is performed by the Anti-Spam channel, using Isode developed technology. Isode spam checking does not use Spamassassin or other third party anti-spam technology. The anti-spam channel records its actions in audit logs, and may delete messages or move them from the message queue to the quarantine.
Users and operators may find information about messages in the quarantine
using the MOI, or by email quarantine summaries sent out by the Quarantine
Email Generator process, based on information in the audit database.
A user or operator can use MOI to mark a message in the audit database
to be released from quarantine. The Quarantine Release Process monitors
for messages to be released, and does this by re-submitting the quarantined
message.
The Anti-Virus channel operates in a similar manner, except that checks
are performed using a third party AV checker. It is straightforward
to use most AV checkers in conjunction with M-Switch.
Message Archive
Messages may be archived at the same time as they are placed into the M-Switch message queue. Messages are archived in a single file per message, in a file system directory. The archive directory is named using date/time, and changes at configurable intervals. This enables the archive directory to remain a reasonable size and to facilitate offline backup. The archive file name is audit logged along with other standard message audit information.
Message Tracking from MOI or MConsole can use information in the audit database to locate messages, and then use the SOM protocol to retrieve archived messages via the QMGR (which directly accesses the identified file in the archive), and display or forward them. Similarly, these tools can also view messages in the queue and quarantine.
Message Delivery Options
As well as standard delivery to M-Box using LMTP, M-Switch supports a number of other message delivery options. Messages may be delivered using a Local Channel to local mail files, either in standard UNIX formats or using the popular maildir format. Delivery time processing includes a sophisticated delivery processing mechanism, controlled by a special language which defines the processing based on a wide range of message parameters. This can be used for personal functions such as automatically filing messages on delivery or sending holiday notifications. It is also useful to support system delivery functions.
The Program Channel (or Shell Channel) enables delivery of messages to arbitrary programs. This can be used to easily integrate message delivery with special purpose delivery functions. This gives a general purpose mechanism for building system specific delivery functions and automatic mailboxes (e.g., “echo” or “delete”).
Housekeeper Channel
A number of general functions are handled by a special Housekeeper Channel, including:
- Processing Delayed Messages. During message submission, DNS and LDAP are used to determine routing and processing of messages. In the event of a timeout on one of these services, the message is “delayed” and message submission is completed later by the Housekeeper Channel.
- Message Warning and Timeout. When a message has been in the queue for a configurable period, it will be timed out, and a delivery status notification sent to the sender. At intermediate times, messages warning of delay may also be sent.
- Operator requested Message Deletion and Non-Delivery. In normal processing, message deletion is handled by the last channel doing work on the message. The operator may request message deletion or non-delivery using MConsole, which will be done by the Housekeeper Channel.
- Message Redirect. The operator may request redirection of a message recipient to a new recipient using MConsole, which is done by Housekeeper Channel.
- Message Reprocessing. Routing configuration for M-Switch may change in a way that would cause different routing for a message. The operator may request that a message or groups of messages are “reprocessed”, so that the updated routing calculation is applied. This is particularly important if a routing change is made to enable queue messages to take an alternate route.
Header and Message Content Processing
Internet messages are structured according to the MIME (Multipurpose Internet Message Extensions) Standard defined in RFC 2045. M-Switch has a MIMEShaper channel that understands MIME format and can perform conversion on messages and components within messages.
Individual body parts can be converted from one to another using conversion filters. Typically this is used for converting a text body part from one character set to another. The necessary conversions are calculated when a message is first submitted and they may be re-evaluated when a message is 'exploded'.
It is often desirable to rewrite header information in particular to
'normalize' addresses by rewriting the address in some canonical form,
rather than one of the multiple addresses that can be used to reach
a specific recipient. MIMEShaper provides options for the header normalization
of Internet message headers are provided. This capability can be used
to provide a coherent view of addresses for local users, or to manage
addresses to give an external view in a firewall type configuration.
Message Priority
M-Switch handles messages based on priority, which is available for all protocols (not just X.400). Message priorities for Internet messages are determined from MIXER "Priority:" Headers. The List Channel may downgrade messages to a "bulk" priority, and Anti Spam channel may downgrade messages to "junk" priority. M-Switch has a number of capabilities for handling messages of different priorities, described in more detail on the M-Switch X.400 product page.
Authorization & Routing Policy
M-Switch provides a comprehensive authorization framework, for controlling which messages can be sent. This includes controls for individual senders and recipients. Controls are often applied on the basis of channel. So far channels have been described in functional terms. Multiple logical channels can be created using the same protocol. For example to create “SMTP Internal” and “SMTP External” channels, each with a group of associated MTAs. This logical grouping can be used to set up authorization policy, which will enable appropriate message relaying and block other types of access.
Security and Authentication
Isode uses Transport Layer Security (TLS) for data confidentiality and Simple Authentication and Security Layer (SASL) for authentication. SASL is also used to map simple identifiers onto directory names for authentication. X.509 (PKI) based authentication can be used, by using TLS and SASL-EXTERNAL authentication. SASL and TLS are supported in the SMTP and LMTP protocols. This is particularly important for controlling message submission, and may also be configured for server to server communication.
SASL and TLS are used by Isode’s SOM protocols, so that secure management from MConsole is possible.
Latency
M-Switch has been designed to give low transfer latency as well as high message throughput.
Internet Mail Conformance
| RFC 1123 | Requirements for Internet hosts - application and support, R. Braden, October 1989 |
|---|
| RFC 1652 | SMTP Service Extension for 8-bit MIME transport, J. Klensin, N. Freed, M. Rose, E. Stefferud, D. Crocker, July 1994 |
|---|
| RFC 1870 | SMTP Service Extension for Message size Declaration, J. Klensin, N. Freed, K. Moore, November 1995 |
|---|
| RFC 2920 | SMTP Service Extension for Command Pipelining, N. Freed, September 2000 |
|---|
| RFC 2033 | Local Mail Transfer Protocol, J. Meyers, October 1996 |
|---|
| RFC 2034 | SMTP Service Extension for Returning Enhanced Error Codes, N Freed, October1996 |
|---|
| RFC 2045 | Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies, N. Freed, N. Borenstein, November 1996 |
|---|
| RFC 2046 | Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types, N. Freed, N. Borenstein, November 1996 |
|---|
| RFC 2047 | (Multipurpose Internet Mail Extensions) MIME Part Three: Header Extensions for Non-ASCII Text, K. Moore, November 1996 |
|---|
| RFC 2049 | (Multipurpose Internet Mail Extensions) MIME Part Five: Conformance Criteria and Examples, N. Freed, N. Borenstein, November 1996 |
|---|
| RFC 2476 | Message Submission, R. Gellens , J. Klensin December 1998 |
|---|
| RFC 2505 | Anti-Spam Recommendations for SMTP MTAs, G. Lindberg, February 1999 |
|---|
| RFC 2821 | Simple Mail Transfer Protocol, J. Klensin (ed), April 2001 |
|---|
| SMTP Service Extensions for Transmission of Large and Binary MIME Messages, G. Vaudreuil, December 2000 |
| SMTP Service Extension for Secure SMTP over Transport Layer Security, P. Hoffman, Feb 2002 |
| RFC 3461 | Simple Mail Transfer Protocol (SMTP) Service Extension for Delivery Status Notifications (DSNs), K. Moore, January 2003 |
|---|
| RFC 3462 | The Multipart/Report Content Type for the Reporting of Mail System Administrative Messages, G. Vaudreuil, January 2003 |
|---|
| RFC 3848 | ESMTP and LMTP Transmission Types Registration, C. Newman, July 2004 |
|---|
| RFC 4468 | Message Submission BURL Extension, C. Newman, May 2006 |
|---|
| RFC 4954 | SMTP Service Extension for Authentication, A. Melnikov, R. Siemborski July 2007 |
|---|
Availability
M-Switch is available on Solaris, Windows, Linux and HP-UX. More details on supported platforms and versions can be found here.
