Messaging and Directory Server software used around the world in the Government, Military, Aviation and Commercial sectors.

isode.com / whitepapers /

Military Messaging Over Low Bandwidth Networks

22nd June 2006

Overview

Military Messages often need to be transferred over low bandwidth networks such as HF radio and other “constrained communication channels”. The two military specifications which have evolved to deal with such a messaging environment are the CCEB (Combined Communications-Electronics Board – AU, CA, NZ, US, UK) developed ACP 142 and NATO's STANAG 4406 Annex E.

This paper describes scenarios that require these special technologies, and then gives an overview of the technologies and how they address the technical problems. Then the paper describes Isode’s approach to implementing these protocols, and how this addresses basic and advanced operational problems, management approaches and integration with other components as part of a larger solution.

 

Click here for a PDF version of this whitepaper.

 

The Technical Challenges

There are a number of basic technical challenges that arise from military messaging deployments, some of which are particularly relevant to constrained communications channels. These are summarized below and will be illustrated in the scenarios described subsequently.

1. Low bandwidth. Many of the communication channels used are very slow, down to as little as 300 bits per second. With bandwidth this constrained, it is imperative that protocols make efficient use of it.
2. High latency. Very often, slow links have long round trip times. Satellite links are faster, but have very high latency. To work well in high latency environments, protocols should be “non blocking” as much as possible.
3. High error rates. Typical communication channels will often have high error rates, and applications must be robust to this.
4. Multicast. Many of the communication channels used are inherently multicast (e.g., radio, satellite). Messages are often sent to multiple destinations, and it is desirable that protocols can take advantage of the multicast nature of the underlying media. To some extent, this can compensate for low bandwidth.
5. EMCON (Emission Control). Deployed units will in many situations wish to not broadcast signals, in order to help hide their location. The situation where signals can be received but not sent is referred to as EMCON. It is important to be able to send messages to a unit in EMCON.
6. Priority. Formal military communications have an associated priority (precedence). In a low bandwidth environment, it is easy for message queues to build up, and so it is critical to have mechanisms which will ensure that the highest priority messages get through first.

Scenarios

This section considers a number of scenarios where the technologies described here are important. It is not intended to be an exhaustive list.

Surface Fleet Communication

Naval communication is a major target. Although communication could flow from the strategic environment directly to task force ships using broadcast radio, this is not generally the approach used. When ships are deployed as part of a task force, communication will generally go to the designated command ship (usually a larger surface unit with the necessary command, control and communication equipment).

At the strategic level, messages may come from a COMCEN (Communications Centre) on shore or directly from originators in the strategic environment. Maximum support of messages direct from originators is desirable.

Messages are then relayed onwards to the other ships in the task force, often using the same communications technology as in ship to shore. The above scenario shows use of Satellite to reach the command ship and broadcast HF (high frequency) radio for communication with the other units in the task force. Multicast can be used for communication from the command ship to the other ships. In EMCON, messages can be received from shore or from other vessels, but not transmitted between ships or back to shore. More complex situations may arise with individual vessels in EMCON.

In general, a ship would have quite a number of potential internal message recipients, and the external message communication needs to be connected with the internal messaging infrastructure.

Submarine Communication

Communication with submarines introduces a number of special requirements. Submarines may make use of higher bandwidth channels when they are on the surface. They will also use VLF (very low frequency) radio, which has a data rate of around 300 bits per second. VLF radio has the advantage that it will penetrate below the surface for a moderate distance, and so can be used by a submarine without surfacing.

When submarines dive below the level of VLF penetration, they will be out of all communication, and so this leads to three basic states: full communication; EMCON; no communication. These need to be managed, and shore systems will be most effective if they understand the current state of communication, a situation also relevant to the previous sacenario. This is achieved by planned timing of communication status, which is shared between submarine and shore.

Army Deployment

The army has similar requirements. Typically, there will be high bandwidth communication to field HQ. Communication to field units may be bandwidth constrained, and there may be requirement for EMCON. In some situations, store and forward messaging is useful, for example to send key information (e.g., map data) or to send a formal message in preference to voice communication.

In the scenario illustrated above, there is no direct communication from field HQ to a second field unit., but messages can be relayed by use of the messaging system in the first field unit. This relay needs to be fully automated, and not require manual intervention at the first field unit.

Special Forces

Another requirement is to support special forces operatives. It will often be desirable or essential to have radio silence (EMCON), but to retain the ability to listen and to receive messages.

Drivers for Change

Military formal messages over low bandwidth networks are generally sent today using ACP 127, which is a text based system. There are two main reasons that formal messaging is moving on from this older technology:

1. NATO nations are moving to an X.400 (ACP 123 & STANAG 4406) based system for formal messaging, and all components need to be migrated to become a part of this new system.
2. ACP 127 is text only, and there is a requirement to sent other types of information, such as images (for example maps or photos) or documents.

This white paper describes the technology that is being mandated in future procurements.

Technology Summary

The overall layered architecture for the solution is illustrated above. The components and layers of this architecture are described in more detail in the following sections.

Radio and Satellite Networks

This paper is not about radio and satellite networks (the typical constrained communication channels) and related technologies. The focus is on the messaging applications that use them. The characteristics of these underlying communication channels and integration are critical to the overall solution.

A common underlying protocol is STANAG 5066 “Profile for Maritime High Frequency (HF) Radio Data Communications.” STANAG 5066 is used as an example communications technology in this white paper. Other underlying technologies will be used including:

• Different protocols to support different underlying technologies.
• Vendor proprietary protocols and vendor extensions to STANAG 5066.
• Future NATO standards.

There are two basic approaches to supporting applications over the underlying communications channels, one using IP (Internet Protocol) and one without. These are illustrated below:

STANAG 5066 defines how to support IP (Annex F.10), as shown in the left hand figure above. Direct use of STANAG 5066 (without IP) for military messaging is defined in Appendix A of Annex E of STANAG 4406.

Not using IP has the advantage of removing a protocol layer, and thus improving bandwidth utilization by a (generally) small amount.

Using IP has a number of advantages:

1. Application is decoupled from the router using a standard protocol interface, and so the application and router components can be decoupled. This architectural advantage leads to the other advantages described.
2. Procurement is simplified, as application and channel can be selected separately (mix and match).
3. Multicast can be supported (this generally requires the IP layer).
4. EMCON can be supported (direct application binding will not always support this).
5. Multiple applications can easily share the underlying channel.
6. Migration of applications to new channels is easy.
7. Application support of multiple communication channels is straightforward.

Because of these advantages, this paper is focused on using IP. Isode’s products support use of IP , but have been designed with a clean API, which would enable direct use of the underlying protocol to be supported as illustrated below.

It can be seen that the resulting product would be a close integration between a communication channel product and Isode M-Switch X.400. This could be achieved by the communication channel vendor integrating Isode’s M-Switch X.400 product as a part of their solution.

IP Integration & Routers

The approach to integrate an application with two communication channels using IP is illustrated above. All of the components talk IP, and will typically be connected by a local network such as Ethernet when the components are not operating on the same system.

The application will be an IP end point. The communication channels will be interconnecting with other systems, and so generally a router will be integrated with the communication channel, so that appropriate packets will be correctly routed over the communication channel. In order to support multicast applications, it is necessary to support multicast at the IP level. Where multicast is needed, the routers must be multicast capable.

ACP 142

ACP 142 “P_Mul – A Protocol for Reliable Multicast Messaging in Constrained Bandwidth and Delayed Acknowledgement (EMCON) Environments” is a CCEB standard for multicast and EMCON support, specifically designed to support NATO’s STANAG 4406 Annex E, described below.

P_Mul is an end to end protocol, that makes use of the Internet Standard UDP (User Datagram Protocol) over IP. It works to transfer data reliably from one system, to one or more recipient systems. A brief summary of how it works is as follows:

1. It works out the IP address to use for the set of intended recipients. There are three options:
   • Single recipient (Unicast). ACP 142 is fundamentally a multicast protocol, and unicast is a special case. For Unicast, the standard IP address of the recipient is used.
   • Static multicast. Here a multicast IP address is assigned to a fixed set of recipients. This is useful for very small networks, and for frequently used combinations of recipients in large networks. A static multicast address can be used without any special negotiation.
   • Dynamic multicast. Each sender has a set of IP multicast addresses reserved for dynamic multicast. ACP 142 allows the sender to negotiate a specific set of recipients to be associated with one of these addresses. This allows dynamic multicast to be used for an arbitrary set of recipients.
2. The sender breaks up the data to be sent into a fixed number of packets, and communicates the number of packets to be sent.
3. The sender then starts sending out data packets, at a rate appropriate to the underlying communication channel.
4. In non-EMCON, each recipient will communicate back to the sender a list of packets that it has not received. This will allow the sender to retransmit lost packets, and to efficiently complete transfer of the data to all recipients.
5. In EMCON mode, a recipient will not be able to send any data back to the sender. In this situation, the sender will simply retransmit the entire message at intervals, to maximize the likelihood that all packets are correctly received.
6. ACP 142 is aware of STANAG 4406 (six level) message priority. The priority is passed down to the packet level, and higher priority packets are always sent first. This means that a higher level message will naturally “overtake” a lower priority message that is partially transmitted.

The details are more complex, but the essence of how ACP 142 works is quite straightforward. It can be seen that ACP 142 provides the core EMCON and multicast functionality needed.

STANAG 4406 Annex E

STANAG 4406 uses standard X.400 communication over high bandwidth channels. Communication between two MTAs (Message Transfer Agents) makes use of the X.400 P1 protocol operating over “full stack” which includes OSI Session and Presentation layers, and Internet Standard ITOT (ISO Transport over TCP) mapping onto TCP (Internet Transmission Control Protocol) over IP.

STANAG 4406 Annex E specifies operation over ACP 142. MTA to MTA communication still uses X.400 P1, but replaces the “full stack”. Annex E provides a number of capabilities:

1. The “full stack” between P1 and IP is replaced with a simple protocol that provides the necessary services over ACP 142, and minimizes overhead. This provides a block of data to ACP 142 that encapsulates the X.400 P1 information.
2. Annex E provides general purpose data compression, that helps reduce data transfer volume for protocol, addressing information, and general data transferred (e.g., text). This compression complements application specific compression techniques (e.g., map and image compression).
3. Adapts to distributed operation procedures for an X.400 MTA, to correctly integrate with EMCON and multicast.

The key functions of Annex E are to reduce to a minimum the amount of data transmitted, and to integrate ACP 142 multicast and EMCON functionality into an X.400 MTA.

LMTA and TIA

Annex E defines protocols and procedures for integrating an X.400 MTA with ACP 142. Annex E defines two basic configurations of MTA.

1. LMTA (Lightweight MTA). This is an MTA where the only external communication makes use of P1/Annex E. An LMTA is appropriate for a ship where all internal communication goes direct to the LMTA.
2. TIA (Tactical Interface Agent). This is an MTA which makes use of P1/Annex E to communicate with LMTAs (or other TIAs). It will also communicate with “full stack” P1 to other MTAs, to enable LMTAs to be interconnected to a general X.400 network.

Isode's M-Switch X.400 can act either as an LMTA or as a TIA. This is a configuration choice, and there is no product difference between TIA and LMTA.

Supporting Small Systems

For a large unit, such as a ship that has multiple users, it is natural for the system to contain an MTA (LMTA or TIA). The MTA will enable local message distribution and give a natural external interface. For small systems, typically supporting only one user, Isode recommends the following architecture:

This approach uses an LMTA on the same machine as the mail client. The LMTA would have a very simple local and routing configuration. The benefits of this approach are:

1. There are no changes to the mail client needed to support a bandwidth constrained environment.
2. The LMTA is optimized to handle the constrained communication channel, and can efficiently manage retransmission and other procedures.

For a modern platform, and efficient MTA such as Isode M-Switch X.400, there is little functional or system overhead in having an MTA on the local system.

Isode Solution

Isode provides ACP 142 and STANAG 4406 Annex E as a part of its M-Switch X.400 product. This section gives a high level overview of the implementation, and the key design features.

Solution Overview

The above diagram illustrates how the relevant core components of M-Switch X.400 operate. The central part of an MTA is the queue of messages in disk. Reliable store and forward messaging provided by an MTA works by receiving a message and reliably storing it in a local queue. The message can then be sent onwards (or delivered locally) and then it is removed from the queue. Even where a message is only held for a few hundred milliseconds, this reliable storage is central to MTA function.

M-Switch X.400 has a queue manager process (“QMGR”) that is responsible for scheduling and control of MTA operations. QMGR interacts with “channels” that do the work of sending and receiving messages. The X.400 Channel is used for handling “full stack” X.400 P1 communication over high bandwidth channels.

ACP 142 and Annex E support are provided by a new ACP 142/X.400 channel that provides support for transferring X.400 P1 over ACP 142. This fits cleanly into the M-Switch functional architecture.

ACP 142 Channel Operation

The ACP 142 channel is a static process, which listens on a UDP port for incoming packets. This is essential, as incoming data may arrive at any time. The ACP 142 channel can access the MTA queue, and it also has its own disk cache that contains information that will optimize performance and allow correct operation in the event of a system restart. In particular:

• Status of all transfers (send and receive).
• Pre-computed data packets to send.
• Received data packets from partial transfers.

The ACP 142 channel will gather incoming packets, and report transfer status to the QMGR. When a complete message is received, it will be submitted into the M-Switch queue. When a message is to be routed over ACP 142, the QMGR will instruct the ACP 142 channel to process the message, which will be in the MTA queue. The ACP 142 channel will then pre-compute the data packets and send the message. The ACP 142 channel will inform QMGR of transfer status, and update information in the MTA queue when needed.

Configuration Management

Isode’s approach to configuration management is to hold information in a directory, as illustrated above. In most deployments, a directory server such as Isode’s M-Vault will be operated on the same server as M-Switch X.400, and this can be considered to be a part of M-Switch X.400. Directory based configuration allows for secure client/server configuration management, and facilitates sharing of configuration information between multiple MTAs.

M-Switch X.400 and the ACP 142 Channel access the directory using LDAP (Lightweight Directory Access Protocol).

Isode provides GUI management of M-Switch X.400 using its EMMA product (Enterprise Messaging Management and Administration).

EMMA (Click for more detail)

This includes configuration of Annex E and ACP 142 configuration. EMMA accesses the directory using X.500 DAP (Directory Access Protocol). DAP is used, as this is mandated in some military environments, and DAP supports security features not available with LDAP. Three screen shots of EMMA, showing ACP 142 GUI configuration are shown below.

EMMA P-Mul In Tab (Click for more detail)	EMMA P-Mul Out Tab (Click for more detail)
EMMA P-Mul Parameters (Click for more detail)

Isode publishes the directory schema used for its configuration, so configuration changes can be made by third party directory clients. This allows for full configuration management to be handled by different tools, or for selected changes to be made by specialized tools. The benefit of both of these options is explained later.

Operational Management

 

The diagram above illustrates the M-Switch X.400 operational management structure. The QMGR process is at the centre of operational control. QMGR interacts with the ACP 142 channel for both incoming and outgoing messages. When a message needs to be sent over ACP 142, QMGR requests “start transfer” to the ACP 142 channel, which will then begin to process the message.

QMGR has the option to pause or abort transfer of messages. This can be in response to specific operator control, as a consequence of QMGR intelligent scheduling, or in support of general events such as “minimize” where only messages above a certain priority are processed.

The ACP 142 channel will periodically report to the QMGR the state of transfer of both inbound and outbound messages. This will ensure that QMGR is aware of the status of all partially complete inbound and outbound transfers.

Management applications interact with the QMGR using the Isode SOM (Switch Operational Management) protocol. Isode provides 'C' and Java client APIs to SOM, to enable third party management applications to work with QMGR. Isode provides the MConsole GUI, as a part of M-Switch X.400 which allows operators to view message and queue status, and perform control operations, such as pausing transfer, requesting non-delivery, and redirection of messages. The following screenshots show of the capabilities of MConsole.

(Click for more detail)

This screenshot shows a message arriving. As the message has not been decoded, only basic information is available.

(Click for more detail)

(Click for more detail)

These screenshots show a queue of messages for Annex E / ACP 142 delivery, the first shows message-level status and priority, the second transfer status and percentage of message transferred.

(Click for larger image)

This screenshot shows a queue of messages for “full stack” X.400 P1. Note that this gives a very similar view – management of a TIA will provide a similar and familiar interface for both sides.

Audit, Tracking and Statistics

M-Switch X.400 records its activity into an audit log. These audit logs are processed and placed into an ODBC audit database. Isode publishes the schema of this audit database, so that third party applications can analyze the data, for example to generate reports.

Isode provides a number of applications based on the audit database, in particular to enable message tracking, and to provide statistics. Some sample statistics are shown below.

Message distribution

Message latency

 

Communication Channel Integration Points

M-Switch X.400 will generally be part of a larger system, and so good integration is critical. The basic integration approach with communication channel components is use of IP. This could be replaced with direct integration with the communication channel, as described above.

Moving a system into EMCON requires changes to both the communication channel and to M-Switch X.400. By holding EMCON status as a directory attribute, it will be straightforward to integrate the M-Switch control into a separate application, which will enable the integrator to provide a simple EMCON control so that the operator can switch both components from one place.

Bandwidth and retransmission parameters are also externally configurable. In a TCP system, a good TCP implementation will automatically adapt its parameters to the performance of the underlying channels. This is not so easy for ACP 142, as it will not receive information to enable this to happen, particularly in EMCON mode. Changes in the underlying communication channel may lead to increase or decrease of available bandwidth, as would other applications sharing the same underlying channel. It would be desirable to integrate this control with the underlying communication channel and directory based configuration enables this.

Messaging Integration Points

Isode’s management architecture is client server, using open protocols where possible. Where Isode proprietary protocols are used, Isode provides client APIs for external integration. In particular:

1. Configuration management is all handled in the directory, with published schema, so that external applications can manage configuration.
2. Operational management uses the SOM (Switch Operational Management) protocol, for which Isode provides client APIs.

These APIs have been used by several Isode partners to produce integrated applications. Two are of particular interest for military deployments.

Insider Technology’s Sentra product provides advanced integrated management for messaging applications. Sentra makes use of SOM, LDAP, and analysis of Isode audit logs to provide a range of advanced management functionality. Sentra can be used to provide an integrated management view onto Isode M-Switch X.400 and onto Microsoft Exchange.

Boldon James are providing an integration of M-Switch X.400 and its Annex E/ACP 142 capabilities with Microsoft Exchange. This provides a tight integration between M-Switch X,400 and Exchange using Isode’s X.400 Gateway API. Configuration is stored in Microsoft Active Directory. Management is provided with a GUI that is closely integrated with Microsoft Exchange management (using Microsoft Directory management APIs and SOM), to provide a solution that will be straightforward for Exchange administrators to handle.

Benefits of the Integrated ACP 142 Architecture

A key Isode design decision is to tightly integrate ACP 142 with M-Switch X.400, as opposed to an ACP 142 implementation that is decoupled from the core MTA with a separate queue. This has a number of advantages:

• ACP 142, Annex E and X.400 have closely related functionality, and close coupling gives a more efficient and robust overall implementation.
• Configuration of ACP 142 parameters, can be naturally associated with the equivalent X.400 MTA components in a single directory location.
• Operational management can be integrated. A decoupled ACP 142 implementation will lead to two queues, with a message being present in both queues at the same time. The two separate queues need to be managed, and will most likely have different management abstractions.

These advantages can be clearly seen from the Isode configuration and operational GUIs, illustrated in earlier sections.

Handling Multiple Communication Channels

The paper so far has primarily considered use of a single constrained bandwidth communication channel. This section shows that many real deployments have multiple communication channels, and explains how the Isode products deal with this.

Scenarios and Requirements

A ship will typically have two or more communication channels: HF Radio and Satellite. The combinations in use will depend on threats:

• Both channels in EMCON.
• Both channels active.
• One channel in EMCON. Typically Satellite channel will be in EMCON, because it has a stronger signal and is more visible. However, the “pencil beam” nature of the satellite and location of threat may lead to HF channel in EMCON and satellite active.

When both channels are open, use of Satellite may be preferred (higher bandwidth) or use of HF radio (lower cost), or a more complex preference dependent on messaging load.

With a headquarters unit, the EMCON status is simpler (as it would never be in EMCON), but there may be many more channels, for example to support multiple satellites with different ships reached through different satellite services. Other shore based systems, such as Special Operations could be in EMCON.

It can be seen that these multi-channel scenarios add some significant complexity.

Network Level Configuration

Given the use of IP, a natural approach to handling multiple networks is to use a single ACP 142 channel, and IP level configuration as shown above. In this structure, an MTA has a single ACP 142 Channel, connected to the two communication channels. IP routing is then used to control which of the channels is used. This is a clean and simple approach with easy switching between channels, including switching channels mid-message.

However, it has a number of disadvantages:

• ACP 142 parameters, such as transmission rate and retransmission timers cannot be tuned for both communication channels at the same time.
• Where messages, or specific classes of message (e.g., high priority) need to be sent over a specific channel, this cannot be configured.
• Dealing with multicast, where connectivity of the two channels is different is likely to be very complex to configure.

For these reasons, MTA level configuration as described in the next section is usually preferable.

MTA Level Configuration

Configuration can be managed at the MTA level, by making use one channel for each communication channel. ACP 142 parameters can be tuned appropriately for each communication channel.

M-Switch X.400 routing can be used to select an appropriate channel for each recipient. In the event of needing to change channels while messages are queued, this can be handled by routing table modification, followed by reprocessing the queue. This will cause transfers on the first channel to be aborted, and then the messages will be moved to the other channel and queue.

Multi-Hop Routing

The initial scenarios showed situations where a message needs to be sent from TIA to LMTA and then on to another LMTA. This multi-hop routing can be important where there is not direct connectivity from TIA to an LMTA. Isode’s directory based routing is based on RFC 1801 “MHS use of the X.500 Directory to support MHS Routing”. RFC 1801 defines the concept of “routing tree”, which is routing information shared by one or more MTAs. Use of multiple routing trees enables complex multi-hop routing to be configured, while still allowing common routing information to be shared between MTAs. This section shows how various approaches to deploying Isode’s routing can be used to support different scenarios.

Isolated Directory Routing

In this basic scenario, each MTA has a directory that holds routing information for each MTA. This might be considered a similar model to each MTA making use of local routing tables. The directory is a local system to the MTA.

This approach will often be a sensible approach for managing large LMTAs and TIAs, so that routing can be configured locally, without any need for directory traffic between the servers.

Shared Routing for a TIA

An alternate setup is for two MTAs to share routing information in single (logical) directory. This configuration has the advantage of needing to manage routing information in one place. For resilience, a directory server will generally be run on the same server as each MTA. As the routing information is the same for both MTAs, the configuration will be replicated using X.500 DISP. This will give efficient incremental changes, and will generally be set up for on demand replication.
This approach would be suitable for a TIA implemented in a distributed manner with multiple MTAs having shared configuration. Managing common configuration in a single location is an important advantage of a directory server approach.

Remote Configured Routing for a small LMTA

The example above shows a third variant, where MTA 1 and MTA 2 have independent routing configurations, but both configurations are mastered and managed on the directory server associated with MTA 1. The configuration for MTA 2 is replicated onto the directory server associated with MTA 2.

This approach will be sensible where MTA 2 is a small unit, where the local staff do not have sufficient skill or time to manage the routing configuration. For such a unit the routing configuration is likely to be fairly small and stable. In the event of changes, these will be made remotely, at a location where there is skill to handle the updates. Then the changes will be replicated when there is network availability and bandwidth.

Conclusions

This paper has set out the STANAG 4406 Annex E and ACP 142 technologies, how they are used to support military messaging, and Isode’s implementation of these technologies as a part of its M-Switch X.400 product.