Messaging and Directory Server software used around the world in the Government, Military, Aviation and Commercial sectors.

isode.com / whitepapers /

Meta-Directories: Cutting Through the Hype

Summary: Meta-directories (techniques for providing directory integration) have been proposed by a number of directory vendors as the solution for 'unifying' and centrally managing disparate directories within an enterprise. This white paper discusses the concept of meta-directories, and examines all the primary techniques available for handling and managing multiple directories. This paper argues that although these techniques are valid and useful, grouping them together and referring to them collectively as a 'meta-directory' is misleading and can have serious consequences.

Share this whitepaper:

The Meta-Directory Vision

The vision of 'meta-directory', as put forward by its advocates, is fairly simple. Most large organizations have many disparate directories which demand significant resources to maintain and manage. A meta-directory product will, in theory, provide seamless integration of all these directories within the organization.

Although this 'seamless integration' would be very desirable, in practice it is extremely difficult to achieve for the following reasons:

1. Although meta-directories are usually described in very simple terms, with broad statements about the problems that will be solved, the problems are extremely complex with many difficult issues to resolve.
2. There is still tremendous confusion about what a meta-directory actually is, as different vendors approach the directory integration problem in different ways. Organizations frequently implement meta-directory solutions without realizing the limitations and scope.

The rest of this paper describes the 'disparate multiple directory' problem and looks at the techniques which can be used to solve it.

Integrating Directories in Practice

Directories are a very important element of the modern IT and telecommunications infrastructure. They are used to provide a range of services, from mail address book information, to printer service locations, to Public Key Infrastructure (PKI) support.

Because of the range of functions, and different driving applications, most organizations end up with many different directories. The requirement is to avoid duplication of management and services, and maintain coherent directory integration.

The three techniques currently proposed to help solve directory integration problems are described below:

1. Integrating directories into a single distributed directory service.
2. Directory synchronization.
3. Loose directory interconnection.

Integrating Directories into a Single Distributed Directory Service

This technique uses directory protocols (typically LDAP) to integrate the various directories as a part of a single distributed enterprise directory service.

The concept of an 'Enterprise Directory Service', accessed by use of directory access protocols (including LDAP, CLDAP, and X.500 DAP), is central to the vision of the directory enabled enterprise. A key part of this model is that the directory service may be provided in a distributed manner using multiple directory servers, with the directory data partitioned and replicated around the enterprise. This high level of distribution enables localized administration, service resilience against network outages and effective scaling. Distribution can be achieved naturally with LDAP/X.500 enterprise directories, and is a key benefit of directory service provision.

The distributed provision aspect of directory services has been played down by a number of vendors of 'LDAP Servers', which are essentially centralized servers (with limited capabilities for distribution and replication) that can be accessed by LDAP. Use of LDAP in this manner leads to an enterprise having a series of independent 'LDAP accessible' directories. These 'simpler' directories, which are described by vendors as providing a 'directory service', are in reality providing a number of independent directory services.

One meta-directory approach, using this technique, is simply to take a number of disjointed directory services, and to integrate them together as part of a total single distributed directory. (Typically this requires functionality to connect simple LDAP servers into a full enterprise directory.) Integrating directories in this way is often a good thing to do, however as the result of this integration is an enterprise directory service, introducing a further name to describe this approach is merely confusing.

Directory Synchronization

A looser way to link directories is to perform directory synchronization; to exchange data between directories to some extent, and provide varying levels of integration. Whenever there are functionality or data structure (schema) mismatches, however, this will cause loss of user functionality and operational headaches. Where the functionality of the two directory services is similar, and data can be mastered in both directories, directory synchronization will end up as an ad hoc solution that will be provided often, but generally unsatisfactorily. Where the information is less related, directory synchronization may be a more useful technique to keep different services aligned. Most meta-directory products have directory synchronization as a core element. This usually comprises two basic types of function:

1. Operations to enable two directories to maintain synchronization, by sharing information between them.
2. Operations to maintain data synchronization between a directory and some external database or other data source (e.g. HR System).

Loose Directory Interconnection

The final solution is to provide a much looser structure to enable users to access multiple directories. This approach abandons trying to provide a coherent single service, and works to give the user a reasonable view onto different services. This is the essence of the World Wide Web approach, and is highly successful for many purposes, for example for the more 'user information' oriented directory services. The limitations of this approach will prevent movement forward on key directory applications, however, such as support of messaging, PKI, and single sign-on.

The Problem with the Meta-Directory Concept

The above techniques are valuable for solving real multiple directory problems. The basic concern with the term meta-directory is that labeling these techniques collectively is not useful. There are a number of reasons for this:

• The term sets false expectations. Although meta-directory sounds like a new tangible service, it means in practice that various existing directories are linked together by various mechanisms, often implemented as a series of ad hoc techniques.
• It causes confusion because it can mean very different things. The resulting meta-directory service will depend on which techniques are used, and the overall meta-directory could be a single distributed directory service, multiple directories, or a WWW interface.
• It distracts organizations from the primary issues of directory service provision, solving short term problems without adequately considering long term requirements.

Isode believes that the concept of the meta-directory does not add value, and it is better to consider each of the approaches to handling multiple directories individually, without grouping them together with an 'umbrella' tag.

Suggested Approach to Multiple Directories

Isode believes that the first thing organizations should consider is their long term directory requirements. A distributed enterprise directory is a key enterprise service, with clear benefits as described in the Isode White Paper Why Deploy an Enterprise Directory?

The second step is to define the options which can deliver the long term vision, considering all of the existing directories and other related information sources which are needed now and will (at some stage) migrate to the long term vision.

There are three basic options (described earlier) that can be used:

• Single directory integration: merge the existing directories as a part of a single enterprise directory.

  This approach would involve taking each existing directory service, and integrating it as a part of another directory service. The result of this is that all the directory services simply behave as a single directory, and from the user standpoint the directories are merged into one.

• Deploy directory synchronization: allow multiple directories to exist, with information synchronized between them.

  This approach would use techniques to share data between directory services. The result of this is that multiple directory services share common data. The user perception of this is that each of the directories contains the same information - typically in this situation a user will only connect to one of the synchronized directories.

• Loose directory interconnection: Provide a common user interface to multiple directories.

  This approach would recognize that there are multiple directories with different information contained in them, and it would provide the user with convenient mechanisms to access all services independently (e.g. using WWW).

For a large organization, with many directories, it is quite likely that a combination of these techniques will be used.

The Isode Solution

This section considers how the Isode products can be used to provide solutions for organizations dealing with the problems of handling multiple directories.

Integration into a Single Distributed Directory Service

Isode provides a market leading LDAP/X.500 Enterprise Directory Server, which is designed to support the core of a large scale distributed enterprise directory. This is used primarily to provide a long term solution. The distributed system capabilities of this product, including support for X.500 replication and directory system protocols, make it ideal to form the backbone of an enterprise directory. This style of solution is described in the Isode white paper How to Build an Enterprise Directory with LDAP and X.500.

As well as providing the core enterprise directory backbone, the Isode directory server is designed to integrate in LDAP only services as a part of the enterprise directory backbone. There are two complementary techniques to do this. In each case, the core directory service is configured to know the location and name of the LDAP servers being integrated.

1. LDAP referrals. With this technique, the backbone directory server returns an LDAP referral to the client asking the query, pointing the client to the correct server for the question being asked.
2. LDAP chaining. Here, the backbone directory server connects to the LDAP server on behalf of the client.

LDAP referrals are supported in the current Isode product, and LDAP chaining will be available in the next release.

Synchronization

Directory synchronization has a wide spectrum of functionality. The Isode product set has a number of features to support this, including powerful scripting interfaces (using the Tcl language) and support for LDIF (LDAP Data Interchange Format) for import and export of bulk data. For simple synchronization, Isode recommends use of these basic tools in conjunction with scripts written for the specific synchronization. This approach is simpler and cleaner than using a third party product, which can add even greater complexity. Most directory products provide flexible functionality for import and export of data. Where the data transformation is simple, it is straightforward to directly synchronize two or more directory services.

For complex synchronization, Isode recommends use of a product such as the 'MaXware Directory Data Manager (MDDM)', available from MaXware (www.maxware.no/). This product has high flexibility for dealing with complex synchronization situations, involving many data sources, two way data mappings, and complex data transformations.

Directory Interconnection

Isode views that the World Wide Web is the ideal framework for providing 'loose' interconnections. To enable integration of directory servers into the Web, Isode's Web-Directory Access Server connects a WWW interface onto a directory service. Use of this technique with multiple directories gives a straightforward 'loose' interconnection.

Summary

Meta-directories have been proposed as the complete 'solution' to the 'directory integration problem', however Isode believes that this concept can be significantly misleading and confusing. Isode recommends that the term 'meta-directory' is not used, and that organizations should think in terms of their directory service provision requirements and then consider the three techniques outlined earlier to adopt the most appropriate approach to deal with the multiple directory issues.