Connector Integration Framework

The Java Connector Integration Framework is typically deployed as a standalone executable like the agents for batch workflows in DirX Identity. In this type of deployment, the framework executable is started by the C++-based DirX Identity server, reads its input requests from a file and writes the response to another file. The following figure shows a typical import/export deployment, where the framework acts as an executable that cooperates with the DirX Identity meta controller (metacp).

In this deployment, metacp exports data from the identity store (an LDAP directory) and transforms it to LDIF change format. An LDIF reader within the framework converts the data to SPML requests. The connector carries out the request at the target system and returns an SPML response, which LDIF writer converts to LDIF content format. metacp then transforms the response and writes it back to the identity store. Note that transformation can occur within metacp and also within the framework in both the input and output direction. However, it is metacp that determines the requests: add, modify or delete an entry.

A pure framework export scenario is slightly different, and is shown in the following figure.

In this scenario, the administrator prepares an SPML request document that contains just a search request. The framework’s SPML reader passes this request to the connector, which presents it to the target system. The rest of the procedure operates just as the import/export scenario.

The following command line invokes the framework as an executable:

java -cp classpath siemens.dxm.connector.framework.AgtSessionExe -c confFilename -p programName -v version -d buildDate -Enc \{ATTRIB_ADMIN_PW | ADMIB_PW | NONE} -Timeout timeout -Port port -CryptlogLevel cryptloglevel -AuditLevel auditlevel

where

The standalone executable framework deployment can itself be deployed as a SOAP client that sends SPML SOAP requests over HTTP and receives SPML SOAP responses from a SOAP service. To deploy the framework in this way, just select the appropriate class as the connector. You have two options:

The following snippet shows a configuration with the latest connector:

The proxy class sends the SPML requests to the URL specified in the connection section of this configuration. It must be adjusted to the configuration of the server side. See "Deploying the Framework as a SOAP Service" for an example.

The framework can also be deployed as a SOAP service within a Web container. When deployed as a Web service, the framework receives SPML SOAP requests via HTTP and returns SPML SOAP responses, as shown in the following figure.

A connector implementation can be deployed as a standalone agent (see Framework as Executable) and in the IdS-J server. In the server it can run as part of a workflow in a number of threads at the same time. This requires the connector implementation to be thread-safe. This means especially to be careful with static variables and in the open() and close() methods. Especially keep in mind that this method pair could be called multiple times within a job!

The interface siemens.dxm.connector.DxmConnectorCore includes the mandatory methods to read the configuration and open and close the connection to the target system:

The interface siemens.dxm.connector.DxmRequestor includes the methods to read, update and delete an entry:

The interface siemens.dxm.connector.DxmConnectorExtended extends the DxmConnectorCore interface and includes the additional optional methods of the DirX Identity connector interface:

DirX Identity defines some conventions and extensions that a connector implementation should follow. The next sections explain these items.

This section provides information about how to use the connector interface that may be valuable for developers of connectors or transformers, including how to:

DirX Identity also provides the Java source for a sample connector that you can use as a template. The sample connector contains examples for logging and reading or writing attribute values and shows how to walk through the attribute lists (as used in Add requests and Search responses), modification lists and search filters. The sample source code is available on your DVD in the folder:

Additions\SampleConnector\java

The following files are available:

The interface siemens.dxm.connector.framework.filter.ConnectorFilter includes the following mandatory methods to read the configuration and open and close the filter.

The interface siemens.dxm.connector.framework.filter. ConnectorFilterConfig includes the optional method to get access to the connector configuration.

The <job> element represents a DirX Identity job and has no attributes. It allows the following subordinate elements:

Within a job configuration there is only one controller element. It represents the controller implementation. An equivalent configuration would use the <connector> element with the role="controller" attribute. The <controller> element allows the following subordinate elements:

It supports the following standard attributes:

The identity integration framework currently provides the following controller implementation classes:

The <operation> element controls the operational options of the job controller. The options that are evaluated depend on the controller implementation. The pass-through controller (DefaultControllerStandalone) evaluates these options for checkpointing.

In general, the operation element accepts the following subordinate elements:

It does not currently support any standard attributes.

The <userhook> element specifies a user extension called a "user hook" that is called during job processing.

The administrator can provide a JavaScript (or a Java class in future releases). Selected job controller implementations (currently SetPassword join engine, error activity, join account) call them at dedicated points while processing an SPML request. For information about how to write a user hook, see the samples delivered with the release.

The <userhook> element accepts the following subordinate elements:

The <userhook> element allows the following standard attributes:

Here is a typical user hook configuration:

The <logging> element specifies the logger and some of its options, especially the log level.

It allows the following subordinate elements:

It supports the following standard attributes:

The logger accepts the following property elements:

timestamp

If set, each log entry is prefixed with a timestamp according the format:

<property name="timestamp" value="EEE MMM d HH:mm:ss.SS yyyy:" />

where EEE denotes day-of-week, yyyy the year, MM the month, d the day, HH:mm:ss hour, minutes and seconds and SS milliseconds.

The format string is evaluated by the JDK class java.text.SimpleDateFormat.

modules

Provides a list of the classes to be logged. Enter the full class names. Wildcards can be included, as shown in the following example:

If not set, the log level applies to all classes.

The <port> element combines a list of <filter> elements with one <connector>. It specifies the port to a target system and is needed when SPML requests from the controller to a connector must be filtered. The <port> contains the configuration for the connector and the filter(s). In this case, the connector must be configured subordinate to the <port> element and not subordinate to the <job>!

The <port> element does not support any standard or custom properties, but just the following sub-elements:

A filter intersects the data flow from the job controller to a connector. Typical implementations encrypt or decrypt SPML requests and responses or build the delta between two consecutive search requests.

The configurations for the filter and the corresponding connector are subordinate to a <port> element. Thus different filters can be specified for different connectors. Several filters can be concatenated. The order of the configuration defines the sequence in which they are processed.

The <filter> element supports the following standard attributes:

The <filter> element only supports the standard sub-elements to specify custom single- and multi-value properties:

Each <connector> element represents a connector implementation such as a reader, writer, controller or target system connector identified by its role name. It allows the following subordinate elements:

It supports the following standard attributes:

Some framework classes accept properties beyond the standard ones. They are entered as <property> sub-elements in the following format:

<property name="some property name" value="the value for your property"/>

The classes are:

List of role names currently evaluated:

Each <connection> element provides information for a connector to connect to the data source: a file, a directory, a database or the like. It allows the following subordinate elements:

It supports the following standard attributes:

This element has the same attributes and subordinate elements as the connector element. The framework provides an XSLT transformer. It expects SPML requests, applies an XSL stylesheet on each request and passes the result back to the framework. The controller (specifically, the default controller class siemens.dxm.connector.framework.DefaultControllerStandalone) hands the transformed request to the next configured transformer or to the connector, if there are no more configured transformers.

The transformer is configured as shown in the following snippet:

This element has the same attributes and subordinate elements as the connector element. The framework provides an XSLT transformer. It expects SPML responses, applies an XSL stylesheet on each response and passes the result back to the framework. The controller (exactly: the default controller class siemens.dxm.connector.framework.DefaultControllerStandalone) hands the transformed response either to the next configured transformer or to the response writer, if there is no more.

The transformer is configured as shown in the following snippet:

The <mvproperty> element is used to define agent-specific multi-valued attributes. It supports the following attributes:

The option values are entered as <value> subordinate elements:

The <property> element is used to define agent-specific additional attributes. It supports the following attributes:

The class siemens.dxm.connector.framework.soap.SpmlSoapProxy is a connector that sends the SPML request as a SPML/SOAP request message to the URL specified in the configuration. The message conforms to the SPML/SOAP binding standard.

The SPML SOAP client sends SPML SOAP requests over HTTP to the configured endpoint and receives SPML SOAP responses from a SOAP service. To deploy the framework in this way, just select the appropriate class as the connector. You have two options:

The connector evaluates the following standard and non-standard properties.

Standard attributes:

The SOAP connector evaluates the following non-standard properties beneath the <connection> element:

Here a configuration sample using the url property to denote the SOAP endpoint:

The following is an alternative with the older SPML implementation using the properties server, port, path and ssl to denote the SOAP endpoint.

The framework class siemens.dxm.connector.framework.LdifChangeReader reads LDIF change requests from file. It does not evaluate any connector attribute or property, but needs the following standard attribute in the <connection> element:

The LdifChangeReader class accepts the following non-standard properties in the <connection> element:

This sample illustrates how these properties function:

Using this configuration, the reader transforms the DN value:

"cn=John Doe, cn=PowerUsers,…​ "

to the SPML identifier "John Doe".

The framework class siemens.dxm.connector.framework.SpmlFileReader reads SPML requests from file. It does not evaluate any connector attribute or property, but needs the following standard attribute in the <connection> element:

and the following non-standard properties (<property name="…​" value="…​"/>):

Sample configuration snippet:

The framework class siemens.dxm.connector.framework.test.SpmlLoopFileReader reads SPML requests from file. In contrast to the SPML file reader, this connector processes the input file in a loop.

In the <connector> element, it evaluates the following non-standard property:

Furthermore, it evaluates the following standard attribute in the <connection> element:

and the following non-standard properties (<property name="…​" value="…​"/>):

Sample configuration snippet:

By using a placeholder in the input file you can arrange for different data per loop.The reader replaces a variable ${counter} with the value of the respective loop.Here a sample snippet for an input request:

The framework provides two response writers that write SPML responses to a file:

siemens.dxm.connector.framework.DxaSpmlFileWriter: SPML response conforms to the latest OASIS SPMLv1 standard (http://www.oasis-open.org/committees/download.php/4138/os-pstc-spml-schema-1.0.xsd).

siemens.dxm.connector.framework.SpmlFileWriter: produces SPML output according to an older version of SPMLv1 that is not considered the official release any more (http://www.oasis-open.org/committees/download.php/2396/cs-pstc-spml-schema-1.0.xsd).

The writers do not evaluate any connector attribute or property, but the following from the <connection> element.

Non-standard properties:

Sample configuration snippet:

The framework class siemens.dxm.connector.framework.LdifFileWriter transforms SPML responses to LDIF content format and writes them to the configured file. It does not evaluate any connector attribute or property, but the following attributes and properties from the <connection> element.

Standard attribute:

Non-standard properties:

<connection …​

<property name="NamingAttribute" value="employeeNumber"/>

</connection>

dn: employeeNumber=123,…​

The LDIF writer processes SPML responses as follows:

The framework provides a connector just for testing purposes: siemens.dxm.connector.framework.TestConnector. The test connector outputs SPML requests to file and returns responses taken from a response file. If a reference request file is configured, it compares each request with the reference and logs errors in case of mismatch. A wildcard "*" in the reference value matches with all values. It does not evaluate any connector attribute or property, but the following attributes and properties from the <connection> element.

Standard attribute:

Non-standard properties:

Here is a sample configuration snippet for the test connector:

The framework provides a response writer just for testing purposes: siemens.dxm.connector.framework.test.SpmlTestWriter. The test writer outputs SPML responses to file. If a reference file is configured, it compares each response with the reference and logs errors in case of mismatch. A wildcard "*" in the reference value matches with all values. It does not evaluate any connector attribute or property, but the following attributes and properties from the <connection> element.

Standard attribute:

Non-standard properties:

Here is a sample configuration snippet for the test writer:

As mentioned in an earlier topic, you insert a C/C++ connector into the C/C++ connector server as a dynamically-loaded shared library. The connector itself is a class inside this library. A connector library is a special case of the connector server component library. It is a shared library that contains one dynamically-loaded component (in this case, the connector class).

The component library must observe the following rules:

The library can contain any number of other classes or functions in addition to the connector class.

To compile the component library, you include the main API header file (which includes other header files needed for the library) and link it with the API shared libraries – see the list of files in "Additional Information". Compile and build the library using compatible compiler and linker environment and settings with the connector server. Note that these implementation details may change with newer versions of the framework.

The connector is an instance of the connector class which must be contained in and exported from the connector shared library. The connector class must be inherited from the abstract class CConnectorInterface. It provides the following methods:

In addition to the connector class, the connector shared library must contain the implementation of the following CConnectorInterface methods:

Note: depending on the connector configuration, more than one connector class instance may exist at one time, each instance running independently in a separate thread. If you intend to configure the connector class this way, appropriate mechanisms should be used when implementing the class to ensure the multithreading capability of the class.

The instance of the class CConfigData is given as an argument into the connector’s open method. This class is called the connector configuration class and is declared in the configuration header file (see "Additional Information" for a list of relevant files). It contains "get" methods for retrieving particular configuration parameters that are in effect for the current connector (like user, password, type, name, version, server, port) and for retrieving properties, which are a set of name/value pairs. See also the contents of the configuration header file.

There are two ways to react to errors inside a connector or a filter:

If you fail to catch all exceptions inside the connector or filter, the server will consider any thrown exception as a fatal C++ exception (because the server has no way of determining the cause of the exception). In this case, the server will place the connector in the error state and it will not be operational until the server is restarted or reconfigured.

The connector server contains a framework for convenient logging of application messages. This framework is accessible for the connector implementation using the macro DXM_MSG_PRINTF. As arguments for this macro you must insert:

The logged messages will be written into the DirX Identity C++ server log.

Memory management is one of the most important quality aspects for C++ applications, and memory leaks represent the worst problem in this field. Memory leaks are pieces of memory that were allocated using various allocation mechanisms (new, malloc, STL allocators and so on) but which were not released once they were no longer needed. In a continuous-run application like the connector server, memory leaks cause it to consume all operational memory of the computer on which it’s running and then crash. As a result, you need to develop connectors and filters that do not contain memory leaks. The connector server API offers a heap check mechanism for debugging memory leaks. It is intended for classes defined inside the connector library (and not for classes from third-party libraries like STL).

The connector and filter interface classes are inherited from the CCSvrObjBase class, which implements the heap check mechanism. When allocating instances of a class inherited from CCSvrObjBase, you must use the macro CSNEW instead of the standard new operator. For deallocating (destruction) of such object instances you use the normal delete operator. The heap checking mechanism records all of these allocations and logs the object instances that have not been deallocated at the end of the server run. In a normal server termination, these instances represent memory leaks. You can obtain the line number at which the allocation occurred from the server log – logging component csvr, level debug 1.

The heap checking mechanism is turned off by default so that it does not adversely affect the server’s performance. You can turn it on for debugging purposes; see the server-wide configuration section for details.

There are two groups of ISR classes: compound classes and SPML classes. The compound classes are wrappers for an SPML request/response pair. In normal operation, the server receives an SPML request and reacts with an SPML response, while the connectors actually handle the SPML data object - the requests are the input data for the connector and the responses are the output data. However, responses are logically bound to the requests. They constitute one logical data entity (which can be described as the data of one connector transaction).

From the logical point of view, the connector receives (in the call to modify, add, delete or other interface methods) one instance of the compound ISR object. This compound object contains only the SPML request object. The connector handles the data from the request and then stores the results of its operation in an SPML response object and inserts it into the original compound object. This object is then returned to the server. One operation of the connector can thus be described as reading and modifying the received ISR compound object and acting according the information in it.

Examine the base compound class CRequestResponse. Its relevant methods include:

Other compound classes are derived from the base class. These derived classes implement the particular main SPML operation types. Look at the class CModifyRequestResponse. it contains the following methods (in addition to the inherited methods from the common compound base class):

The SPML classes provide a C++ implementation of the standard SPML classes, and include the following types:

Look at the base classes. The class CSpmlBase is the base class common to all SPML classes (request, response and substructure classes). The following methods are of interest:

The class CSpmlReqRespBase is the base class common to all request and response classes and inherits the methods from CSpmlBase. The following methods are of interest:

There is also a base class common to all request types (that is, not responses). This class CSpmlRequest inherits from CSpmlReqRespBase and has only some minor "get" methods. For responses, there is the CSpmlResponse base class, which inherits from CSpmlReqRespBase. It contains methods for handling SPML result and error codes.

Request objects (for example, CModifyRequest) inherit from CSpmlRequest and contain additional methods for handling special SPML substructures contained only in the respective type of request. For CModifyRequest the Modifications element is an example. Handling of other SPML subelements is inherited from CSpmlRequest and CSpmlReqRespBase respectively. Analogous with the response classes (e.g. CModifyResponse).

The ISR classes are normally created within the C++ connector server after the SPML requests are received from various communication protocols. Once a request is received, it is unmarshalled into the request type and inserted into the appropriate compound object (for example, CModifyRequestResponse).

The compound object is then sent to the connector methods (Add/Modify/Delete/Search). The rest of this section describes the actions that should be performed inside the connector to handle the request correctly and return the correct response. Here the actions will be explained in the context of a modify request, the use of other request types is analogous.

First you must get the request from the compound object, for example, by calling GetModifyRequest. From the CModifyRequest pointer you read the SPML data that the connector needs for its operation. This is the request ID (for logging purposes), the identifier, operational attributes and modifications. You use the "get" methods of the request class (CModifyRequest) or its base class methods. These methods often return pointer to sub objetcs (for example, CSpmlAttributes). You should not modify or delete these objects that you get from the request object. These objects are only meant to be read.

After getting the information, you handle the actions in the connector; that is, write some data into the target system, change or delete them. The results of this data operation must be written into the response object.

First you must create a response object instance inside of the compound ISR object. For this action, call the CreateResponseFromRequest method. Subsequently, you can get the pointer to the response object (by calling the GetModifyResponse method) and use the Add/Set/Copy methods of the response class or its base classes to add the data to the response.

You should insert the results of a correct connector operation as well as the error information in the response. In case of a serious exception, you can also throw an exception; see the section "Handling Exceptions".

The request and response instances must remain in the compound object. With the GetModifyRequest and GetModifyResponse, you only receive pointers to these instances so that you can read and modify them. The instances are returned within the compound object after the Modify (or other) connector method returns. Do not delete any SPML object instances you receive from the compound object from inside the connector. They all remain inside the connector and are deleted by the server later on.

The response classes provide several methods for adding or modifying the contents of the object. They copy some substructures from other objects (mainly from the requests), add new substructures or set them to a given value.

The methods are generally used to add new substructures to the SPML objects. Sometimes, however, replacing information inside the SPML objects may be required. In this case, you should be aware of the Set/Add/Copy methods' replace semantics. Some of the methods expect to be used to add a new content to the object and not to replace the old content, because replacing the old content could be unintentional and could lead to information loss. These methods will not allow you to directly replace the contents. You should first explicitly remove the substructure from the SPML object and then add a new instance (otherwise you incur an exception). In other cases, the implicit replace semantics are allowed (where it is natural or does not impose a risk for unintentional data loss).

To determine a method’s replacement semantics, consult the comments at its declaration

Note that all character values (strings) that the SPML and ISR class methods return are encoded in ASCII or UTF-8. Similar if you set values you must use ASCII/UTF-8 encoded character values. You cannot use ISO8859-1 (latin-1) or any other encoding. There is, however, a standard filter defined for converting some encodings (or you may write your own filter for that).

The following options control the connector server as a whole. They are stored with one exception in the directory and consist of the following data:

The connector server uses the same logging mechanism as the DirX Identity C server. Each logged message is assigned to a particular component and a particular logging level. You can specify which components and logging levels inside them should be logged in the logging configuration file. Separate levels exist for errors and warnings and several debug logging levels are used for debug messages. The following components are relevant for the connector server: cutl, csvr, cwrp, cmgr. "cutl" is a core component of the server; it handles various low-level tasks like thread synchronization or dynamic library loading. "cmgr" is a module for reading configuration data from LDAP or files. "cwrp" is a module that controls the activity of connectors. "csvr" is the central server module that controls the network interface as well as the flow of data through the server.

Connectors log their messages as ’conn’ components where n represents the logging number of the respective connector. See the description of connector configuration.

The following options are available for every configured connector:

Some rarely-used configuration options are configured in the C++ DirX Identity server’s INI file:

The current DirX Identity connector server version was compiled using Microsoft Visual Studio 6. It uses the STLport implementation of the STL library (www.stlport.com). Because STL objects are part of the connector server API, it is necessary to compile the connectors using STLport as well. To find out how to compile and link your source with this library, see the product related documentation. You must ensure that the standard Microsoft STL header files are ignored and instead the STLport headers are included. The same applies to the STL runtime libraries. Note: Do not forget to use the namespace "_STL" instead of "std".

When compiling a DLL on Windows, you must typically define a preprocessor symbol for the DLL linkage (is usually defined in the header file and differently redefined for the import / export case). As a result, the main API header file contains the symbol DXMCCONINTERFACE_EXPORTS. This symbol should be defined when compiling the connector library.

You also need to include some standard compilation symbols. You can find them in the sample connector implementation discussed in the section "Sample Implementation".

Use the "Multithreaded DLL" library.as the C runtime library.

You can also compile the server under Visual Studio .NET 2005 environment as an unmanaged C++ application. This environment has its own multithreading-capable STL implementation so you won’t need to use the STLport implementation.

This document refers to the following files: