Sodium SyncDirectory and Data Synchronisation
Sodium Sync enables synchronization between directory servers and other data sources such as files and databases. Sodium Sync incorporates extensive functionality addressing the complexities encountered when synchronizing data from multiple sources and in scenarios which include constrained bandwidth, transferring data across secure boundaries, at firewalls with 'air gap' requirements and across data diodes.
Originally designed as a directory synchronization tool working to and from Isode's M-Vault, Microsoft's Active Directory and other LDAP or X.500 directory servers. It has evolved into a comprehensive data synchronization tool with extensive data transformation, correlation and merging capabilities.
Sync Configuration, Scheduling and Workflow
Syncs are configured and scheduled using a Wizard interface which offers immediate access to three common directory to directory sync profiles, four common LDIF transformations as well as access to an Advanced Wizard view giving fine-grain control of the synchronization process. Simple syncs occur as independent events but more complex scenarios exist where it makes sense for syncs to have relationships to each other and to external events. Sodium Sync allows for the grouping of syncs and external events into a Directory Synchronization Workflow. For more information see the page on sync configuration, scheduling and workflow.
Data Transformation, Mapping, Merging and Correlation
Sodium Sync incorporates extensive functionality addressing the complexities of data transformation & mapping, merging & correlation encountered when synchronizing data from multiple sources and in certain scenarios. You can read more about this on the page on data transformation, mapping and merging.
Data Format Support
Sodium Sync’s primary goal is to synchronize directories supporting LDAP or X.500 DAP access.
LDIF (LDAP Data Interchange Format) defines a text format for representing directory data. LDIF files will generally be used to hold data corresponding to a directory subtree. Sodium Sync treats an LDIF file as equivalent to an LDAP or DAP directory subtree, and an LDIF file can be used as source or target for a sync. LDIF can be used to represent a set of actions on a directory, which can also be consider as a 'delta'. The term 'change LDIF' is used to distinguish this type of file from an LDIF file that represents a sub-tree. Change LDIFs are used to support Sync by Email. Change LDIFs can be useful for ad hoc directory management, and so Sodium Sync enables loading of change LDIF, and comparison of two sources and generation of a change LDIF to represent the delta.
There is often the requirement to handle data from other sources with directory synchronization, Sodium Sync provides support for data import and export using two widely used interfaces:
- CSV (Comma Separated Value) format files (RFC 4180).
- SQL Databases.
Access, Authentication and Connection Security
Sodium Sync shares directory access with Isode Sodium directory management GUI, and details are shared with Sodium. Sodium offers a number of security options for the two primary protocol access mechanisms (X.500 and LDAP).
|Security Option||X.500 DAP||LDAP|
|Simple Authentication (password)||y||y|
|Strong Authentication (PKI)||y||y|
|TLS data confidentiality||y|
Sync by Email, Air Gap and Data Diode
There are a number of situations where normal directory replication protocols cannot be used. For example:
- In constrained bandwidth environments such as HF radio, where performance will be poor.
- Across secure boundaries, where directory replication and access protocols may not be used.
- At firewalls with 'air gap' requirements.
- Over data diodes.
Sodium Sync provides a number of related solutions for these environments. Email is often a practical communication mechanism when directory replication cannot be used, so directory replication over email is a key building block.
Sodium Sync provides capabilities to generate LDIF changes relative to an automatically stored reference copy. This enables Sodium Sync to generate a sequence of change LDIFs, and for another copy of Sodium Sync to robustly apply them, checking for duplicates, missing updates and out of order changes. It can also drive 'transport' programs before or after it runs. This provides capability to replicate between directory servers using email, or to provide directory replication across an 'air gap' gateway.
More information on synchronization by email and scenarios that require it are given in the whitepaper [Directory Replication by Email and over 'Air Gap']. Information on using this capability with M-Switch is given in the whitepaper [File Transfer by Email]. Sodium Sync can also be used to perform directory replication over a data diode, to support directory replication in secure environments with one way data flow.
Attribute syntax checks and custom data mappings can be defined in XML. XML provides a flexible mechanism for mapping information, that includes selection and transformation of attribute values using regular expressions. When this is insufficient, mappings can be extended by the user of scripting
Performance and Scaling
The basic operational mode of Sodium Sync does a "full update" on each run. This works by reading data from both directories, and then applying any necessary changes to the target directory. This is robust, straightforward, and works with any LDAP or DAP directory. Sodium Sync works by streaming data, and minimizes the amount of data it holds at any time. This enables it to scale to synchronize very large directory information trees. On a modern Core i5 machine, between fast directory servers such as Isode's M-Vault, typical performance for Sodium-Sync is around 300 entries per second. This makes it practical to synchronize several thousand entries with updates at very short intervals.