XMPP has a Publish-Subscribe capability, generally referred to as PubSub, which many XMPP experts see as very important. This whitepaper seeks to explain PubSub and its significance to non-experts. The paper looks at the problems addressed by publish-subscribe systems, shows how XMPP PubSub is used for Military Forms Publishing, discusses PubSub capabilities & potential Applications, and outlines M-Link's PubSub support.
Isode whitepapers are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
XMPP has a Publish-Subscribe capability, generally referred to as PubSub, which many XMPP experts see as very important. This white paper seeks to explain PubSub and its significance to non-experts. PubSub is a building block for other applications and not directly used by human users so, to show its capabilities and benefits, this paper focuses on applications using PubSub rather than the technical details. It also looks briefly at the PubSub implementation provided by Isode's M-Link server products.
This paper assumes basic familiarity with XMPP concepts.
The Publish Subscribe Problem
Publish-Subscribe is a well-known technique, that is often used in large scale distributed systems. Publish-Subscribe is used when there are lots of events happening, and entities interested in differing subset of the events. A mechanism where each entity independently tried to find out about the events of interest would be complex and inefficient; it requires resource polling which is resource intensive and does not scale to very large deployments.
With Publish-Subscribe, publishers and subscribers are decoupled. Publishers classify events into topics and each event is sent to the Publish-Subscribe service. Subscribers indicate to the Publish-Subscribe service which topics they are interested in and receive events associated with each topic as they are sent to the Publish-Subscribe service, giving access to information in real time.
Publish/Subscribe systems are useful for a wide range of applications to provide efficient sharing of data.
Forms are important for military operations, and there is often a need to handle forms quickly and share with a large number of users. XMPP based open standard instant messaging is widely used by military organizations and is a sensible framework for sharing forms.
XEP-0346 “Form Discovery and Publishing” (FDP) has been specified to address these requirements. Templates describing form layout are published using XMPP PubSub, and a stream of forms matching the template may then be published. Those interested in specific forms can subscribe and are given updates when new forms arrive. This provides an efficient and flexible approach to form publishing.
FDP has been developed for military forms, but the capability provided is generic. There may well be other applications, such as first responder support.
Further details are given in the Isode White Paper Military Forms using XMPP
Overview of XMPP PubSub Capabilities
In this section you'll find a high-level overview of XMPP PubSub features (XEP-0060: Publish-Subscribe) without going into detailed explanations.
Use of XMPP Stanzas
PubSub works by exchanging XMPP Stanzas (messages) in and out of a PubSub server. There are four
basic PubSub message types:
- Subscription: A subscriber sends an XMPP iq (information.query) stanza to register a subscription. The user will then receive events from the PubSub server while the subscription is active.
- Publishing: The Publisher sends an XMPP iq stanza to publish an event, typically (but not always) including a payload.
- Events: Events are sent to subscribers as XMPP message stanzas.
- M-Link's PubSub is fully integrated with M-Link Clustering with clustering of the PubSub nodes, so that PubSub operation will continue without interruption in the event of server failure
- Management: XMPP manages the PubSub system as well as providing the server (information discovery and management of the PubSub system are all handled with XMPP stanzas).
Nodes are logical entities within a PubSub server used to group information, they are the entities to which events are published to which content consumers subscribe. In other Publish Subscribe systems they are called 'topics' and this name makes the concept clearer for content syndication services like Collecta and buddycloud.
Nodes are a framework for dispatching new events to subscribers. A node can also hold old events, so that a new subscriber to that node can review historical events. Collecta showed the benefits of this capability.
Nodes can be grouped into flexible hierarchies (technically a directed acylic graph) using collection nodes. Events are always published to leaf nodes and then flow up the hierarchy. The hierarchy gives a convenient way to group nodes in order to help subscribers easily subscribe to a set of nodes or to all nodes.
Use of the node model works well for applications where the data model for matching publication and subscription can be set up separately to data contained in the events. However in both of our examples, Collecta and buddycloud, users subscribe to events based on data in those events. XMPP PubSub supports this by use of a content filter in the subscription.
PubSub provides flexible access control to support a number of broad approaches:
- Open (anyone can publish or subscribe)
- Managed (the PubSub service provider decides who can do what)
- User controlled (publishers and/or subscribers control access)
Personal Eventing Protocol (PEP): An extended XMPP Service built on PubSub
Personal Eventing Protocol (PEP) is an example of an extended XMPP service built on PubSub. PEP is a mechanism to support extended Presence, so that a user can publish information in addition to the core online/offline status, and control how that information is shared. For example, a user can publish location information (GeoLocation) and music being listened to. This information can then be shared with selected members of the user’s roster. This is all built with PubSub, which gives a number of advantages:
- By using a core XMPP mechanism, duplication of low level XMPP functionality is avoided.
- The user can use PubSub access control, to give flexible control of who this information is shared with.
- Roster members will only subscribe to this information if they choose to, and so resource is saved by only sending the data to those who want it.
Multi-User Chat and MIX
Multi-User Chat (MUC) is a widely XMPP service that bears some resemblance to PubSub. It is described here to clarify the relationship, and also to give those familiar with MUC a different way to think about PubSub. You could view sending a message to a MUC group as 'publication' and joining a MUC group as 'subscription' and MUC groups as 'nodes' in a PubSub server. MUC is not built on PubSub, as it was specified before PubSub.
XEP-0369: Mediated Information eXchange (MIX) is a new standard, which Isode is actively involved in developing. MIX is intended as a medium term replacement for MUC and it is based on XMPP PubSub.
Other Services that could use XMPP PubSub
XMPP PubSub has potential to provide support for a wide range of applications. This section looks briefly at some things that might be done, to illustrate the potential offered by PubSub.
The essence of social networking is that individuals make information available to be shared with others. Distribution needs to be controlled based both on controls specified by the person publishing the information and on interest of potential recipients of the information (subscribers). Most desire this information to be available immediately, so an underlying publish/subscribe mechanism is ideal.
Use of an open protocol (XMPP PubSub) would enable sharing across boundaries, rather than constraining social networking to closed systems to which all participants must belong. Social networks such as Facebook and identi.ca are providing XMPP access, and use of PubSub would be a natural extension of this.
Content Syndication and replacing RSS
Content syndication such as Collecta and Google Alerts is an important capability. Use of an open standards infrastructure would benefit sharing.
An interesting observation is that RSS, which is polling technology widely used for content sybdication, is inefficient. It would be highly desirable to replace this with an open publish/subscribe mechanism. XMPP PubSub could be used for this, and it has been suggested that the next generation of Web browsers should provide integrated XMPP support in order to enable this. This is a quite radical idea.
Military Situational Awareness in Constrained Networking Environments
Situational awareness is a vital military application for sharing data between users. There will often be significant volumes of data to share. There may be particular interest in change of location of certain entities, or the availability of photographs. In many deployed environments, there is insufficient bandwidth to broadcast all data. So a publish subscribe system, where a user can subscribe to 'changes
in this area' is ideal.
In an era of partner interoperability with joint operations, use of an Open Standard publish/subscribe base would be ideal.
Event Recording/Alerting Systems
Isode server products generate 'events', which can range from severe operational errors to warnings and diagnostic information. These are specified by comprehensive XML catalogs, and available in log files, syslog and Windows events.
These could be provided using XMPP PubSub, with events delivered to general purpose or special XMPP clients. PubSub could then be used to give user control over which events are received. For example "All severe events on all operational servers and directory replication errors and warnings on test directory servers".
Feedback on the desirability of such a capability, which we are considering implementing, is welcome.
M-Link XMPP PubSub Support
Once a decision has been made to use XMPP PubSub to support an application, a choice will need to be made between various XMPP Servers that support PubSub. M-Link supports PubSub, and has the following benefits:
- M-Link was designed from the start to support PubSub, and so provides excellent performance that does not impact other XMPP functions.
- PubSub is a substantial specification, and M-Link provides a very full implementation, with all major features included.
- M-Link’s PEP implementation is based on PubSub (this is not true of many PEP implementations) and so capabilities of PubSub such as access control are available to PEP users.
M-Link Console provides flexible PubSub configuration and visualisation capabilities.
This paper has given an overview of Publish-Subscribe and shown examples of where this technology can help in providing a range of solutions. The benefits and capabilities of XMPP PubSub as an open standard for Publish-Subscribe have been explained, and the market leading features of the M-Link XMPP PubSub implementation.