Sodium Sync: Data Transformation, Mapping & Merging
Isode's Sodium Sync enables synchronization between directory servers and other data sources such as files and databases.
On this page you can read about data transformation & mapping, data checks and correlation & merging. On other pages you can read about sync configuration and scheduling, how Sodium Sync can combine syncs and data checks into a Directory Replication Workflow, other Sodium Sync features and Sodium Sync Conformance.
A key capability of Sodium Sync is transformation of data between source and target. When synchronizing data, variations in data and schema can lead to a number of requirements, such as:
- Military directories will generally follow ACP 133. In practice different nations will operate services that follow specific national variants.
- Microsoft Active Directory implements LDAP but deviates from the standard in a number of ways.
- Moving data between services often occurs at boundaries leading to requirements to remove data or to map the data to give a different view.
- When data is merged from multiple sources (described below) there needs to be clear control over where specific data comes from to ensure that a given source does not interfere with data from another source.
The entries tab of the Sync Profile Wizard controls what is synchronized within the specified point of the DIT. Filters can be applied to source or target, an LDAP filter can be applied (for example to select only entries of a given object class), entries can be limited to a specific depth and specific subtrees of the DIT can be excluded.
The attributes tab gives control of attribute handling on the selected entries; detailed options can be seen by clicking on the screenshot above. Attribute filters provide a number of actions:
- To allow through a configured set of attributes.
- To allow through all attributes, except for a configured list.
- To delete the entry (block all attributes).
- To control the values of the object class allowed through.
Sync Profile Mapping (click to show/hide)
The Mapping tab gives access to a number of built in mappings and to custom mappings (which can be defined in XML).
The Glue tab gives a number of options to deal with a problem that occurs in some configurations where an entry is replicated and its parent entry is not present (e.g. because it was filtered out). This can be used to remove levels in a hierarchy and 'flatten' the directory tree.
The Checks tab allows the configuration of a number of standard checks:
- Referential Integrity is applied to all attributes with Directory Name syntax, to verify that they point to a directory entry that exists.
- Update size can be constrained, primarily to ensure that unintentional major changes or deletions are not propagated.
- Check that a given attribute is unique, for example to ensure that no two users have the same telephone number. This is important for syncs into Active Directory as email address must be unique for Exchange.
Click on the screenshot below for a clear view of the Checks tab.
Checks Tab (click to show/hide)
Attribute syntax checks can also be configured by defining custom XML. Selection of this text is then done in the mapping tab.
Sodium Sync can be used to provide a simple sync process such as from a sub-tree in one Directory (or other data source) to another Directory. However Sodium Sync's merging and correlation functions allow it to be used to handle more complex setups such as bringing data from multiple different directory services into a single central directory.
While this sort of task can be addressed by setting up a series of independent synchronizations, it's often desirable to merge data. Sodium Sync supports data merging based on a model that every piece of data has a clear master location, ensuring that the overall state can be determined definitively.
- Every entry must have a directory which is authoritative for naming.
- Every attribute within an entry must have a directory which is authoritative for the value of that entry.
A simple merged configuration is shown above with the primary sync controlling name and selected attributes and the secondary syncs handling additional attributes. Attribute filter controls are used to ensure that the syncs handle only the data they are designed to handle.
The setup described above assumes consistent naming between all of the sources and target. As this will not usually be the case, Sodium Sync includes correlation functionality needed to provide consistency between data sources. Correlation is also important when using non-directory sources in order to manage the mapping of that data onto a directory structure.
Managed by a GUI Interface correlation allows a specific 1:1 mapping for data items to be configured. Correlations can be extended by the scripting interface to support specialized situations.
Sync Correlation Report (click to show/hide)
A sync correlation report can be generated (above) which shows:
- Correlated entries.
- Entries that were correlated but have become problematic due to data changes.
- Suggested matches (and misses) for data that is not correlated.
Correlated entries can be ignored (if on one side only), discarded or approved.