Messaging Protocols for HF Radio

Published: 28th August 2008
Last updated: 28th November 2008

Purpose

Messaging is important for military and other deployments of HF Radio. Formal Military Messaging (STANAG 4406) over HF Radio is described in the Isode White Paper Military Messaging over HF Radio and Satellite using STANAG 4406 Annex E. This paper looks at:

How to optimize STANAG 4406 messaging for point to point HF networks.
How to provide Internet Messaging over multi-node and point to point HF networks.

Gaps in the current set of available standards are identified, and this paper describes two new protocols specified and implemented by Isode to fill the gaps.

Current Messaging Standards

	Internet Messaging	STANAG 4406
Point to Point	HMTP CFTP	STANAG 4406 Annex E & ACP 142 (sub-optimal)
Multi-point (Including EMCON)	No Standard	STANAG 4406 Annex E & ACP 142

The table above shows the NATO messaging standards for messaging over HF. Two sets of standards are referenced:

STANAG 4406 Messaging is described in the Isode White Paper "Military Messaging over HF Radio and Satellite using STANAG 4406 Annex E".
HMTP (HF Message Transfer Protocol) and CFTP (Compressed File Transfer Protocol) are defined in STANAG 5066 Annex F. CFTP, despite its name, defines a message transfer protocol. CFTP is sometimes referred to as BFEM (Battle Force Email).

HMTP and CFTP are specified to work over a point to point network - that is, to communicate between a pair of nodes, with both nodes able to send data. A point to point communication cannot take advantage of the inherently multicast nature of HF, and cannot handle a node in EMCON (Emission Control) that is not allowed to transmit data.

STANAG 4406 Annex E with ACP 142 is designed to support multicast and EMCON. While it can work over a point to point network, this is suboptimal. The reasons for this, and why an optimized protocol is important, are described in the next section.

Isode's goal in the work described in the paper is to provide good options for all four quadrants in the protocol diagram above. The rest of this paper describes how we achieved this.

Why Point to Point needs a Different Protocol

HF Radio networks are generally set up to work either point to point or multi-node. Although it would be possible to set up a network that could work both ways, this is not usually done in practice. If it was, the analysis described here would still apply to the different modes of network use.

In a point to point setup, two nodes can talk to each other (i.e., data can be transferred in both directions). This two way communication can be taken advantage of in a number of ways.

Link setup. Modern point to point communication will use Automatic Link Establishment (ALE) and Fast Link Setup (FLSU). FLSU minimizes time to get started. ALE enables two nodes to choose the best frequency to communicate on, and the optimum waveform to use. This leads to an optimized link. Although modified ALE can be used for multipoint networks, in practice multipoint networks used fixed frequency and a lowest common denominator choice bandwidth and waveform to ensure that it works for all parties, noting that some may be in EMCON.
When a variable speed waveform, such as STANAG 4539, is used, the receiver can measure signal/noise ratio and use STANAG 5066 to communicate to the sender to switch to a speed that will give better overall performance. This can only be done point to point.
When a point to point link is used, retransmission of data can be done at the data link layer, typically using STANAG 5066 ARQ or STANAG 4538 ARQ. This gives efficient retransmission, requiring much less data to be resent than would be required for application retransmission. Data link retransmission provides the application with reliable data transfer. This enables two key optimizations for a higher level protocol:

There is no need for application retransmission, as this is handled at the data link layer. This increases efficiency, and removes the latency and timer issues associated with application retransmission.
Application PDUs can be large, as they will not be retransmitted. This will reduce protocol overhead.

All of this gives a clear benefit to point to point communication, and to protocols optimized to use it.

Isode's Protocol Architecture

The above diagram shows Isode's protocol architecture to meet the goals set out above. The key features of this architecture are:

Use of BSMTP (Batch SMTP) as a mechanism to transfer Internet Mail, as an alternative to STANAG 4406.
Introducing a Connection Oriented ACP 142 protocol, to optimize transfer over point to point links.
Mappings onto STANAG 5066 (for HF) and IP (for Satellite).

These features are each described in more detail below.

BSMTP (Batch SMTP)

Simple Mail Transfer Protocol (SMTP) is defined in RFC 2821 and is the Internet Standard for transferring Internet Mail. SMTP is an interactive protocol between a pair of servers. BSMTP (Batch SMTP) is a means of representing an SMTP transfer in a file, originally used for supporting email on IBM networks that did not support TCP. A BSMTP format is specified in RFC 2442. BSMTP works by transferring the file from a sending MTA (Message Transfer Agent) to a receiving MTA. The receiving MTA will process the BSMTP file, with any errors or success notifications handled by sending BSMTP files containing Delivery Status Notifications (DSN) back to the sending MTA.

Isode's BSMTP format is based on RFC 2442. It supports all of the capabilities of RFC 2442, in particular:

NOTARY support, that enables full control of DSNs, in particular to request positive delivery reports and to suppress all DSNs.
Support for 8BITMIME and BINARYMIME formats, which enables efficient transfer of messages containing binary data.

The changes made by Isode are:

Transfer of single message only. There does not appear to be any benefit in batching messages, and single message transfer leads to some simplification of encoding and implementation.
A more compact encoding to reduce protocol overhead.

This BSMTP format is used as an alternative to the STANAG 4406 Annex E message format. STANAG 4406 Annex E extensible compression and transfer syntax is used with BSMTP. This is essential functionality, and it makes sense to use the STANAG 4406 Annex E protocol.

Connection Oriented ACP 142

Connection Oriented ACP 142 is a protocol designed to provide the ACP 142 transfer service for a single destination over a point to point network. It uses that ACP 142 DATA PDU, and introduces a new 'Address and Data' PDU, which uses encodings based on other ACP 142 PDUs.

Connection Oriented ACP 142 is defined within the existing ACP 142 service, so that an application does not need to choose whether to use ACP 142 or Connection Oriented ACP 142 (CO-ACP 142). In general, an application will not be able to make this choice, as it will not know if the destination is in EMCON. So, the application will pass the message to the single ACP 142 service, which will decide whether or not to use Connection Oriented ACP 142.

CO-ACP 142 PDUs have a maximum size of 64 kbytes. A message sent by CO-ACP 142 uses one 'Address and Data' PDU, followed by zero or more DATA PDUs. Message transfer is then acknowledged using a single DATA PDU. The acknowledgment confirms transfer of the message to the peer application, and ensures there is no message loss.

Connection Oriented ACP 142 provides a very simple reliable transfer protocol, that optimizes provision of the ACP 142 service over point to point networks.

Operation over STANAG 5066 and IP

There are two target bearer services for ACP 142 and Connection Oriented ACP 142.

STANAG 5066. This is appropriate for use with HF Radio, as described in STANAG 5066: The Standard for Data Applications over HF Radio.
IP. This is appropriate for use with other constrained links, such as satellite.

ACP 142 is defined for use with generic datagram service provider, and the two target networks are supported by:

UDOP (Unreliable Datagram Oriented Protocol) is defined as a part of STANAG 5066 to provide a datagram service.
UDP (User Datagram Protocol) is a standard Internet datagram service running over IP.

Similar mappings are defined for Connection Oriented ACP 142, which is designed to work of a reliable link layer:

RCOP (Reliable Connection Oriented Protocol) is defined as a part of STANAG 5066 to provide reliable transfer of data over a point to point link, and provides a user mapping with multiplexing and concatenation of the underlying Unit Data service in ARQ mode. This gives an efficient mapping for HF.
TCP (Transmission Control Protocol) gives a stream mapping onto IP. CO-ACP 142 assumes reliability, and there will be complete message retransmission in the event of TCP connection failure. This mapping is suitable for high reliability links, such as satellite.

This architecture means that the protocol family can be used for efficient operation over HF and Satellite.

The New Protocol Matrix

	Internet Messaging	STANAG 4406
Point to Point	HMTP CFTP BSMTP & CO-ACP 142	STANAG 4406 Annex E & CO-ACP 142
Multi-point (Including EMCON)	BSMTP & ACP 142	STANAG 4406 Annex E & ACP 142

A new protocol matrix can now be drawn, including the new Isode protocols. Key features of this new matrix:

The STANAG 4406 / Multipoint solution (STANAG 4406 Annex E over ACP 142) is unchanged.
There are protocol solutions for the two quadrants that the new protocol architecture set out to address:

Internet Mail / Multipoint can now be addressed with BSMTP over ACP 142.
There is an optimized solution for STANAG 4406 / Point to Point (STANAG 4406 over Connection Oriented ACP 142).

A third option has been introduced for Internet Mail / Point to Point (BSMTP over Connection Oriented ACP 142).

CFTP vs BSMTP over CO-ACP 142

It is useful to compare BSMTP over Connection Oriented ACP 142 with the existing standards. HMTP has a number of quite serious deficiencies as a message transfer protocol, and so the comparison in this section is made with CFTP, which is the better protocol and also in wider use.

The benefit of CFTP is that it is a published standard and is in active use. BSMTP over Connection Oriented ACP 142 offers the following benefits:

Can be used over IP, as well as over STANAG 5066, so can be used for Satellites and other environments where STANAG 5066 is not appropriate.
Can switch cleanly to multi-point deployment, or to use of EMCON for two nodes.
Integrates cleanly with STANAG 4406 messaging, with optimized performance, and control of all messages using precedence.
Supports NOTARY, which enables control of Delivery Status Notifications, and in particular the ability to request positive DSNs.
Support 8BITMIME and BINARYMIME which will optimize performance for binary data.
Removes the 128 MByte maximum message size.
Enables higher priority messages to “overtake” lower priority messages that have started transmission. While CFTP can support this, it may only be used with peer agreement (which is a nuisance to configure), and is not generally supported.
Extensible choice of compression algorithm, which provides a framework for significant performance optimization in some deployments, by use of custom compression or custom compression libraries.
Slightly smaller messages, due to a more compact encoding.
Use of MTA to MTA acknowledgments is a cleaner protocol architecture, and protects against message loss in some scenarios.

These benefits are sufficient to justify consideration of using BSMTP over Connection Oriented ACP 142 in preference to CFTP.

What Isode Provides

Isode provides support for STANAG 4406 messaging in its M-Switch product, support for Internet Messaging, and support for MIXER, that enables conversion between the protocols. M-Switch also provides support for ACP 142 as a 'channel', described in "The Architecture of Isode's STANAG 4406 Annex E Solution".

To support the architecture described in this paper, Isode has extended its ACP 142 system, to enable transfer of Internet Messages using ACP 142, and to provide support for Connection Oriented ACP 142 as a part of the ACP 142 channel. This means that Isode can provide the architecture described in this paper in a single product, which will enable flexible deployment of messaging services.

Developing Standards

Isode believes strongly in open standards, and sees them as critical for large scale deployment. We have developed two new protocols, because there are no suitable open standards. We are very happy to provide protocol specification to NATO or to other standardization bodies, and will not assert patent or other IPR rights on the protocol designs.

Conclusions

This paper has shown the need for two new protocol standards to support Internet Messaging and STANAG 4406 over HF Radio, and has described these protocols, which are provided by Isode in its M-Switch product.

Isode Whitepapers

Stay Informed

From the Isode blog...

Creative Commons