Data Format Handling Procedures

This section discusses procedures necessary to handle typical data formats:

  • XML Formats

  • ChangeLog Formats

Handling XML Files

This section describes how to handle XML data files in DirX Identity.Specifically, it describes how to:

  • Read an XML data file

  • Write an XML data file

See also the XML data format section.

Reading an XML Data File

To read an XML data file, perform the following steps:

  • Create the attribute configuration handle ah by passing the name of an attribute configuration file to the meta readattrconf function.

  • For flat XML, create a connection handle ch with a meta openconn function of the form:

    • meta openconn -conn ch -type FILE -format FLAT-XML -attrconf ah -filemode READ …​

  • For DSML: create a connection handle ch with a meta openconn function of the form:

    • meta openconn -conn ch -type FILE -format DSML -attrconf ah -filemode READ …​

  • Use the meta getentry function in a loop for reading the file, for example:

    • meta getentry -source ch -target eh

The Tcl variable array eh is available in the same way as usual and can be used for mapping purposes.

Keep in mind that the enclosing tokens in an XML data file are <xml_data.xml> and </xml_data.xml>, for example

<?xml version='1.0' encoding='ISO-8859-1'?>
<!-- Example -->
<xml_data.xml>
         ...
</xml_data.xml>

Using different tags in the data file results in a warning “Invalid record” in the trace file at run time.

Writing an XML Data File

To write an XML data file, perform the following steps:

Create an attribute configuration handle ah by passing the name of an attribute configuration file to the meta readattrconf function.

  • For Flat XML: create a connection handle ch with a meta openconn function of the form:

    • meta openconn -conn ch -type file -format FLAT-XML -attrconf ah -filemode WRITE …​

  • For DSML: create a connection handle ch with a meta openconn function of the form

    • meta openconn -conn ch -type file -format DSML -attrconf ah -filemode WRITE …​

  • Use the meta gethandle function in to create a handle for the output file, for example:

    • meta gethandle -conn ch -entry eh

The Tcl variable array eh is available in the same way as usual and can be filled for mapping purposes.To write the data XML file from a directory search result:

  • Write the header using the meta writetext function

  • Call the meta getentry function on the directory search result in a loop:

    • Perform the mapping of the directory entry to the entry represented by the handle eh obtained with meta gethandle

    • Write entry eh using the meta writerecord function

  • Write the footer using the meta writetext function.

Handling ChangeLogs

The ChangeLogs are read from the iPlanet or OID directory by calling obj search.For each object in the search result a getentry is called and a TCL variable array with the result is filled (see also changeLog data formats).So for the first example listed above, there might exist TCL variables such as

rh_ldap(DDN) = Changenumber=207, cn=changelogentry
rh_ldap(objectClass) = {top changelog}
rh_ldap(targetdn) = {cn=Shrivastava Saurabh,ou=metadirectory,o=oracle,dc=com}
rh_ldap(changetype) = Add
rh_ldap(operationtime) = 19991029153546z
...
rh_ldap(changenumber) = 207
rh_ldap(orclchangeretrycount) = 0
rh_ldap(changes) = objectclass:inetOrgPerson
          objectclass:OrganizationalPerson
          objectclass:Person
          objectclass:Top
          cn:Shrivastava Saurabh
          cn:Shrivastava
          cn:Saurabh
          cn:Saurabh Shrivastava
          cn:sshrivas
          employeenumber:22
          givenname:Saurabh
          sn:Shrivastava
          title:Mr.X
          mail:sshrivas@us.oracle.com.X
          telephonenumber:650-574-9107
          postaladdress:1067 Foster city Blvd $Apt B$Foster city CA=94404-X
          l:Foster city
          orclguid:0003968769422514087583716869442
          creatorsname:cn=orcladmin
          createtimestamp:19991029153546z

The relevant fields that are of further interest are highlighted.

In order to handle the iPlanet and OID-ChangeLogs, the following operation can be used:

meta getchangelog
          -name <DN>
          -changetype <operation change type>
          -changes <list of changes>
          -source <handle to pseudo LDIF-file>
          -target <result handle>

metacp internally can parse LDIF-Change_Files. As that parser should also be used for handling of the ChangeLogs, an entry from the Search-Result (which represents the ChangeLogs) needs to be mapped to the correct LDIF syntax. The only items of interest are targetdn, changetype, changes.

Therefore the three interface parameters will be concatenated in memory and an appropriate prefix (if necessary) is assigned to each of them. Such a byte stream can easily be handled by the LDIF parser. The only difference is that the parser gets its values from memory and not from file.

The command getchangelog will fill a TCL variable array (named by result handle of parameter -target) the same way as if getentry would have read an LDIF change file.

Example:

After having called ‘getentry’ for the first entry from the search result, the ‘getchangelog’ will be called as follows:

getchangelog
          -name [lindex $rh_ldap(targetdn) 0]
          -changetype [lindex $rh_ldap(changetype) 0]
          -changes [lindex $rh_ldap(changes) 0]
          -source file_ch
           -target rh

The first entry from the examples listed above will be mapped to the following byte stream:

dn: cn=Shrivastava Saurabh,ou=metadirectory,o=oracle,dc=com
changetype: Add
objectclass:inetOrgPerson
objectclass:Person
objectclass:Top
cn:Shrivastava Saurabh
cn:Shrivastava
....

The following TCL variables will be set:

rh(DDN) = {cn=Shrivastava Saurabh,ou=metadirectory,o=oracle,dc=com}
rh(_ldif_opcode) = Add
rh(objectClass) = {inetOrgPerson Person Top}
rh(cn) = {{Shrivastava Saurabh} Shrivastava}

The parameter $rh_ldap(changenumber) holds the last change number. So the last changenumber is always available and can easily be used in the next synchronisation cycle, when the obj search command needs to be called with the following filter:

-filter (&(objectclass=changeLogEnty)(changenumber>=’LAST-CHANGE-NUMBER’))

ChangeLog Sample Code

In the following sample code, only the relevant functions are listed.

The sample code doesn’t include error handling etc. (e.g. if obj search fails). Furthermore, it doesn’t list the functions that need to be called first in order to get the relevant handles that are passed to openconn, e.g. ah or ldap_ah etc.

#
# open a pseudo LDIF data file (That is just a 'dummy' open for the routine 'getchangelog'. so that this routine
# gets a correct source handle 'file_ch')
#
set cmd "meta openconn  -type     FILE \
        -file          \"$agent_data_file\" \
        -mode     READ \
        -format   TAGGED \
        -attrconf ah       \ # attribute handle generated by ‘readattrconf’
                           \ # when reading ‘l-ldifattr.cfg’
        -conn     file_ch"
if {$agent_attr_list != ""} then {
append cmd " -attribute {$agent_attr_list}"
}
exec_cmd $cmd

#
# create a connection handle "change_ch" that requests all the attributes that need to be read from the
# LDAP server in order to get the changelog entry
# The parameter ‘change_attr_list’ contains all the attributes that are read (by ‘obj search’) from the Changelog
# entry: ‘changes’, ‘targetdn’, ‘changenumber’, ‘changetype’, ‘DDN’
#
set cmd "meta openconn -type     LDAP \
        -attrconf ldap_ah       \ # attribute handle generated by ‘readattrconf’
                                \ # when reading ‘l-ldifattr.cfg’
        -conn     change_ch"
if {$change_attr_list != ""} then {
append cmd " -attribute {$change_attr_list}"
if {$process_all_attr == "TRUE"} then {
append cmd " -processallattr"
}
}
exec_cmd $cmd

#
# retrieve the relevant objects from the LDAP server
# (The parameter filter is set as follows:
# set filter {(&(objectclass=changelogentry) (server=$host_name) (changenumber>=$change_number)
(!(modifiersname=$subscriber_dn)))}
# with ‘change_number’ being the relevant change number that has been processed up so far)
#
set cmd "obj search {$base_obj} \
         $subset \
         -conn       change_ch \
         -result     ldap_res"
if {$filter != ""} then {
append cmd " -filter \"$filter\""
}
set status [catch { eval $cmd } result]

set result 0
while {$result == 0} {
#
# extract an entry from the search result
#
set result [exec_cmd "meta getentry -source ldap_res \
                                    -target rh_ldap"]
if {$result == 0} then {
         set max_change_number [lindex $rh_ldap(changenumber) 0]
         # pass the retrieved attributes of the changelog entry to ‘getchangelog’; the TCL variable array named
         # by the ‘-target’ parameter contains all the changes.
         exec_cmd "meta getchangelog     \
              -name     {[lindex $rh_ldap(targetdn) 0]}     \
              -changetype {[lindex $rh_ldap(changetype) 0]} \
              -changes    {[lindex $rh_ldap(changes) 0]}      \
              -source     file_ch        \ # handle to
                                         \ # pseudo LDIF
                                         \ #  file
              -target     rh"
         #
         # now the change log entry is available in the Tcl-Variable "rh"
         #
         ...
}
}