Fine-Grained Patching of Resources

In DXA, we have extended the OData v4 Config Service to support fine-grained patching of resources. This allows users to update specific members of complex objects or arrays such as modifying one object in a collection or adding new items without needing the full current state of the object or array.

To support this, we introduced patch metadata, which can be sent along with the PATCH request. This metadata describes what should happen to each property of the resource that we sent. The metadata is a JSON object, that is part of the resource builder object sent, its structure is described in the schema below.

It is not required to send patch metadata with every PATCH request. If omitted, each property provided will be treated as a full replacement of its current value.

Patch metadata should only be included when fine-grained control over the patching behavior is necessary.

 
{
    "net.atos.dirx.access.apprepo.api.data.Metadata" : {
        "type" : "object",
        "description" : "Application repository metadata",
        "properties" : {
          "resultSchema" : {
            "type" : "string"
          },
          "patch" : {
            "type" : "array",
            "description" : "The map describing the patch",
            "items" : {
              "$ref" : "#/components/schemas/net.atos.dirx.access.apprepo.api.data.KeyOperationPairType"
            },
            "title" : "Patch",
            "x-requiredJapi" : "true"
          }
        },
        "readOnly" : true,
        "title" : "Metadata"
      },
      "net.atos.dirx.access.apprepo.api.data.Operation" : {
        "type" : "object",
        "description" : "Operation describing how the property in the update builder should be merged with the existing value of the property.",
        "properties" : {
          "filter" : {
            "$ref" : "#/components/schemas/net.atos.dirx.access.apprepo.api.data.PatchFilter",
            "description" : "The filter to be used in case of patching of an array of SubType objects. If no filter is specified, internally the IMPLICIT filter is used.",
            "title" : "Filter"
          },
          "type" : {
            "type" : "string",
            "description" : "The type of the operation to be performed with the property. If the type is PATCH, subProperties must be specified.",
            "enum" : [ "addItem", "replace", "remove", "patch", "replaceItem", "removeItem", "patchItem" ],
            "title" : "Type"
          },
          "subProperties" : {
            "type" : "array",
            "description" : "The sub-properties describing the operation to sub-properties of an object",
            "items" : {
              "$ref" : "#/components/schemas/net.atos.dirx.access.apprepo.api.data.KeyOperationPairType"
            },
            "title" : "Sub Properties"
          }
        },
        "title" : "Operation"
      },
      "net.atos.dirx.access.apprepo.api.data.PatchFilter" : {
        "type" : "object",
        "description" : "Patch filter",
        "properties" : {
          "type" : {
            "type" : "string",
            "description" : "The type of the filter",
            "enum" : [ "IMPLICIT", "IDENTITY" ]
          }
        },
        "required" : [ "type" ],
        "title" : "Patch filter"
      },
    "net.atos.dirx.access.apprepo.api.data.IdentityPatchFilter" : {
        "type" : "object",
        "allOf" : [ {
          "$ref" : "#/components/schemas/net.atos.dirx.access.apprepo.api.data.PatchFilter"
        }, {
          "type" : "object",
          "properties" : {
            "identifier" : {
              "type" : "string",
              "description" : "The String identifier of the item to be matched in a collection (either the key or the full string value)."
            }
          }
        } ],
        "description" : "Patch filter that identifies an item in a collection by its id (key), or full string match.",
        "required" : [ "identifier", "type" ],
        "title" : "Identity Patch filter"
      },
   "net.atos.dirx.access.apprepo.api.data.KeyOperationPairType" : {
        "type" : "object",
        "description" : "An entry definition for a Map.",
        "properties" : {
          "key" : {
            "type" : "string",
            "title" : "key"
          },
          "operation" : {
            "$ref" : "#/components/schemas/net.atos.dirx.access.apprepo.api.data.Operation",
            "description" : "The operation describing how to merge the updated property with the original one",
            "title" : "Operation"
          }
        },
        "required" : [ "key", "operation" ],
        "title" : "KeyOperationPairType"
      }
}

Patch operations

The patch metadata is a map of property names to patch operations. With each PATCH request only one operation can be done on each property. The operation can be one of the following types:

  • replace

    • Replaces the value of a property with a new value from the request builder (same behavior as omitting patch metadata).

    • Can be used for all types of properties

    • If a property is present in the request builder but not in the patch metadata, the replace operation is used by default.

  • remove

    • Sets the property to null (i.e., resets to its default value, if any).

  • patch

    • Used to patch sub-properties of a single object (not an array).

    • Must include a subProperties field, which is a map of sub-property names to operations.

  • addItem

    • Adds one or more items to an array.

    • The new items are sent in the request builder as a JSON array (same as in regular updates).

  • replaceItem

    • Replaces an item/s in an array based on key matching; adds it if it doesn’t exist.

    • Only supported for arrays of complex objects (not primitive arrays).

    • Uses x-primaryKey to identify the item in SubType collections.

    • For maps, the key is used to locate the item.

  • removeItem

    • Removes an item from an array using information from the request builder.

    • Only supported for arrays

    • Uses the x-primaryKey for SubTypes in collection to find the item to be removed. (It is sufficient to only provide the properties that are part of the x-primaryKey)

  • patchItem

    • Patches individual sub-properties of an object in an array.

    • Must include a subProperties map.

    • Can only patch one item per operation.

    • Uses x-primaryKey to identify the item in the array.

    • Cannot patch sub-properties that are part of x-primaryKey (these are used for identification only).

Identifying items in arrays

Configuration SubTypes in arrays do not have explicit IDs. Therefore, items are identified using the x-primaryKey defined in the OpenAPI schema of the SubType.

The x-primaryKey is a list of properties that uniquely identify an item in the array. Each item must have a unique combination of these key values otherwise, the patching behavior is undefined.

Filters

Filters are used during patching to locate items in arrays.

Currently, the only supported filter for the Config REST Web Service is:

  • IMPLICIT

    • Uses x-primaryKey to identify items in SubType collections.

    • For other property types, full value equality is used.

    • This is the default filter and does not need to be explicitly specified.

The IDENTITY filter is not supported in the Config REST Web Service. It is used internally for SCIM 2.0 REST Web Service patching of entity resources.

Examples

The following examples show how each operation can be used in a PATCH request.

Replace operation

Request used to replace the property webPepId of the OAuth Server Endpoint.

PATCH /odata4/config/1_0_0/OAuthServerEndpoints('OAuth%20Server')
{
   "webPepId":{
      "key":"DirX Access PEP",
      "type":"net.atos.dirx.access.apprepo.api.config.client.web.WebPep"
   },
   "meta":{
      "patch":[
         {
            "key":"webPepId",
            "operation":{
               "type":"replace"
            }
         }
      ]
   }
}
The request would behave the same if the meta property was not sent, as the replace operation is the default operation for properties without metadata present.

Remove operation

Example below shows a request that would be used to remove the all the conditions for an authorization rule. Note that it is sufficient to only send the meta property with the patch metadata, the rest of the properties are not needed to be sent in the request during remove.

PATCH /odata4/config/1_0_0/AuthzRules('rule1')
{
   "meta":{
      "patch":[
         {
            "key":"conditionIds",
            "operation":{
               "type":"remove"
            }
         }
      ]
   }
}

Patch operation

The patch operation is used to patch individual sub-properties of an object property. When an object property is sent in the update builder without any metadata, the sub-properties that are present in the update builder will be replaced with the new values, and the sub-properties that are not present will be left unchanged. If the patch metadata is sent, the sub-properties can be patched individually, and the operation can be used to specify how each sub-property should be patched.

The request below shows the patch operation would be used to modify the Communication URLs holder of and authentication method, by replacing login URL and removing Prefix for URL definitions. Note that the sub-property loginUrl can be omitted from the metadata, as the replace operation would be used for it by default (since it is present in the builder).

PATCH /odata4/config/1_0_0/AuthnMethods('method1')
{
    "commonUrls": {
        "loginUrl": "new_login.html"
    },
    "meta":{
        "patch":[
            {
                "key":"commonUrls",
                "operation":{
                    "type":"patch",
                    "subProperties":[
                        {
                            "key":"baseHrefUrl",
                            "operation":{
                                "type":"remove"
                            }
                        },
                        {
                            "key":"loginUrl",
                            "operation":{
                                "type":"replace"
                            }
                        }
                    ]
                }
            }

        ]
    }
}

AddItem operation

The example below shows how the addItem operation can be used to add a new allowed authentication method to a PEP configuration object. The result will be that the new authentication method will be added to the list of allowed authentication methods, without removing the existing ones.

PATCH /odata4/config/1_0_0/PlainWebPeps('DirX%20Access%20PEP')
{
    "allowedAuthnMethodIds": [
        {
            "key": "FIDO Authentication",
            "type": "net.atos.dirx.access.apprepo.api.config.authn.method.AuthnMethod"
        }
    ],
    "meta":{
        "patch":[
            {
                "key":"allowedAuthnMethodIds",
                "operation":{
                    "type":"addItem"
                }
            }
        ]
    }
}

ReplaceItem operation

Replace item operation will generally be used to either replace a complex object in an array, or to replace a value linked to a key in a map.

The first example shows, how the replaceItem operation can be used to replace password in a map of Name-Password map in the Kerberos SPN Table.

PATCH /odata4/config/1_0_0/KerberosSpnTables('My-Company%20Kerberos%20SPN%20table')
{
   "namePasswordMap":[
      {
         "key":"HTTP/my-server.my-company.example@MY-COMPANY.EXAMPLE",
         "values":[
            "newPassword"
         ]
      }
   ],
   "meta":{
      "patch":[
         {
            "key":"namePasswordMap",
            "operation":{
               "type":"replaceItem"
            }
         }
      ]
   }
}

The second example shows how we could replace one Authorization tokens of an OAuth Server Endpoint configuration object. Therefore, the member of collection that is sent in the request is considered "full" meaning, all sub-properties that we do not send, are effectively set to default.

Note that in this case, the member of the collection, that is being replaced, is identified by the x-primaryKey property, which is used to find the item in the array. Therefore, we should check the OpenApi specification to see which properties are used to identify the item in the array.

The oauthAuthzTokens property of the OAuth Server Endpoint is an array of net.atos.dirx.access.apprepo.api.config.client.web.oauth.OAuthServerAuthzToken. The definition of the OAuthServerAuthzToken in OpenApi specification looks like shown below (we omit most of the properties for brevity). From it, we can see that the grantTypes and scopes properties are used to identify the item in the array, so we cannot change them in the replaceItem operation, but we use them to match the entry.

{
   "net.atos.dirx.access.apprepo.api.config.client.web.oauth.OAuthServerAuthzToken":{
      "type":"object",
      "properties":{
         "grantTypes":{
            "type":"array",
            "description":"The permitted grant types to which authorization codes / token issuance is restricted. Supported grant types are ``implicit``, ``refresh_token``, ``password``, ``client_credentials``, ``authorization_code`` and ``urn:ietf:params:oauth:grant-type:uma-ticket``.",
            "items":{
               "type":"string",
               "enum":[
                  "authorization_code",
                  "password",
                  "implicit",
                  "client_credentials",
                  "refresh_token",
                  "urn:ietf:params:oauth:grant-type:uma-ticket"
               ]
            },
            "title":"Grant types",
            "uniqueItems":true,
            "x-requiredJapi":"true"
         },
         "scopes":{
            "type":"array",
            "description":"The permitted scopes to which authorization codes/token issuance is restricted.",
            "items":{
               "type":"string"
            },
            "title":"Scopes",
            "uniqueItems":true,
            "x-requiredJapi":"true"
         }
      },
      "x-primaryKey":"grantTypes,scopes"
   }
}

The example below then shows, how this operation could be used to replace item with grantTypes = ["password"] and scopes = ["jwt"] with a new configuration. If the item with these values does not exist, it will be added to the array.

PATCH /odata4/config/1_0_0/OAuthServerEndpoints('OAuth%20Server')
{
   "oauthAuthzTokens":[
      {
         "grantTypes":[
            "password"
         ],
         "scopes":[
            "jwt"
         ],
         "authzCodeValiditySecs":350,
         "accessTokenValiditySecs":5000,
         "signingApplicability":"NONE"
      }
   ],
   "meta":{
      "patch":[
         {
            "key":"oauthAuthzTokens",
            "operation":{
               "type":"replaceItem"
            }
         }
      ]
   }
}

RemoveItem operation

The first example shows how a removeItem operation can be used to remove an item from allowed authentication methods of a PEP configuration object.

PATCH /odata4/config/1_0_0/PlainWebPeps('DirX%20Access%20PEP')
{
    "allowedAuthnMethodIds": [
        {
            "key": "FIDO Authentication",
            "type": "net.atos.dirx.access.apprepo.api.config.authn.method.AuthnMethod"
        }
    ],
    "meta":{
        "patch":[
            {
                "key":"allowedAuthnMethodIds",
                "operation":{
                    "type":"removeItem"
                }
            }
        ]
    }
}

The second shows how the removeItem operation can be used to remove an Authz Token from an OAuth Server Endpoint configuration object. The passed item in the request only needs to include the x-primaryKey properties, the rest of the properties are not needed to match the item in the array. If multiple items are sent, all of them will be removed from the array.

PATCH /odata4/config/1_0_0/OAuthServerEndpoints('OAuth%20Server')
{
   "oauthAuthzTokens":[
      {
         "grantTypes":[
            "password"
         ],
         "scopes":[
            "jwt"
         ]
      }
   ],
   "meta":{
      "patch":[
         {
            "key":"oauthAuthzTokens",
            "operation":{
               "type":"removeItem"
            }
         }
      ]
   }
}

PatchItem operation

One example of using the patchItem operation is to patch the sub-properties of an item in an array of SubTypes, that can be for example the authzTokens property of an OAuth Server Endpoint configuration object and its sub-property authzCodeValiditySecs. The request will locate the item in the array by the x-primaryKey properties, which are grantTypes and scopes in this case, and change the value of the authzCodeValiditySecs property to 350 seconds. The other properties of the item will remain unchanged.

PATCH /odata4/config/1_0_0/OAuthServerEndpoints('OAuth%20Server')
{
   "oauthAuthzTokens":[
      {
         "grantTypes":[
            "password"
         ],
         "scopes":[
            "jwt"
         ],
         "authzCodeValiditySecs":350
      }
   ],
   "meta":{
      "patch":[
         {
            "key":"oauthAuthzTokens",
            "operation":{
               "type":"patchItem",
               "subProperties":[
                  {
                     "key":"authzCodeValiditySecs",
                     "operation":{
                        "type":"replace"
                     }
                  }
               ]
            }
         }
      ]
   }
}

Other way to use the patchItem operation could be to modify the Name-Password map in the Kerberos SPN Table, for a more complex case, where we want to add a new password to the list linked to a key, but we do not want to remove the existing passwords.

PATCH /odata4/config/1_0_0/KerberosSpnTables('My-Company%20Kerberos%20SPN%20table')
{
   "namePasswordMap":[
      {
         "key":"HTTP/my-server.my-company.example@MY-COMPANY.EXAMPLE",
         "values":[
            "newPassword2"
         ]
      }
   ],
   "meta":{
      "patch":[
         {
            "key":"namePasswordMap",
            "operation":{
               "type":"patchItem",
               "subProperties":[
                  {
                     "key":"values",
                     "operation":{
                        "type":"addItem"
                     }
                  }
               ]
            }
         }
      ]
   }
}

Request with multiple operations

The operations can be combined in a single request, to perform multiple operations on the resource. To demonstrate this, we will use the PlainWebPep configuration object.

Below is shown a resource state on the server before the request is sent, we will omit some of the properties for brevity.

{
    "@odata.context": "https://my-server.my-company.example:10116/odata4/config/1_0_0/$metadata#PlainWebPeps/$entity",
    "dp": "net.atos.dirx.access.apprepo.api.config.client.web.PlainWebPep",
    "key": "DirX Access PEP",
    "sslEnabled": true,
    "cacheTimeoutSecs": 1800,
    "clusterGroup": 0,
    "type": "Plain",
    "authority": "dxa-server:1",
    "nonPermitOrDenyMapping": "Deny",
    "xacmlRequestConstructionTemplateId": {
        "key": "Default Request Template",
        "type": "net.atos.dirx.access.apprepo.api.config.authz.XacmlRequestConstructionTemplate"
    },
    "xacmlPdpId": {
        "key": "RBAC Administration",
        "type": "net.atos.dirx.access.apprepo.api.config.authz.pdp.XacmlPdp"
    },
    "requestInjectionTemplateIds": [
        {
            "key": "Authentication Methods",
            "type": "net.atos.dirx.access.apprepo.api.config.subject.external.RequestInjectionTemplate"
        },
        {
            "key": "Assurance Level",
            "type": "net.atos.dirx.access.apprepo.api.config.subject.external.RequestInjectionTemplate"
        },
        {
            "key": "Authentication Time",
            "type": "net.atos.dirx.access.apprepo.api.config.subject.external.RequestInjectionTemplate"
        }
    ],
    "allowedAuthnMethodIds": [
        {
            "key": "Form Authentication",
            "type": "net.atos.dirx.access.apprepo.api.config.authn.method.AuthnMethod"
        },
        {
            "key": "Basic Authentication",
            "type": "net.atos.dirx.access.apprepo.api.config.authn.method.AuthnMethod"
        }
    ],
    "multiPepAssignments": [
        {
            "contextPath": "/odata4/config",
            "port": 11111,
            "pepId": {
                "key": "DirX Access PEP Internal clients",
                "type": "net.atos.dirx.access.apprepo.api.config.client.Pep"
            }
        },
        {
            "contextPath": "/odata4/config",
            "port": 11112,
            "pepId": {
                "key": "DirX Access PEP Internal clients",
                "type": "net.atos.dirx.access.apprepo.api.config.client.Pep"
            }
        },
        {
            "contextPath": "/odata4/config",
            "port": 11116,
            "pepId": {
                "key": "DirX Access PEP OAuth RS",
                "type": "net.atos.dirx.access.apprepo.api.config.client.Pep"
            }
        },
        {
            "contextPath": "/odata4/sysactions",
            "port": 11116,
            "pepId": {
                "key": "DirX Access PEP OAuth RS",
                "type": "net.atos.dirx.access.apprepo.api.config.client.Pep"
            }
        },
        {
            "contextPath": "/scim",
            "port": 11116,
            "pepId": {
                "key": "DirX Access PEP OAuth RS",
                "type": "net.atos.dirx.access.apprepo.api.config.client.Pep"
            }
        }
    ],
    "cookie": {
        "name": "DXASID",
        "path": "/",
        "version": 0,
        "secureEnabled": false,
        "httpOnly": false,
        "persistentCookieEnabled": false,
        "sameSiteFlag": "NotIncluded"
    }
}

With the PATCH request we will want to replace the authority, remove all the requestInjectionTemplateIds, add a new allowedAuthnMethodIds, change the name of the cookie and remove the multiPepAssignments for the port 11116 and context path /odata4/config. We found in the OpenApi, that for the multiPepAssignments the x-primaryKey is contextPath,port, so we will use these properties to identify the item in the array. The request will look like this:

PATCH /odata4/config/1_0_0/PlainWebPeps('DirX%20Access%20PEP')
{
    "authority": "dxa-server:2",
    "allowedAuthnMethodIds": [
        {
            "key": "FIDO Authentication",
            "type": "net.atos.dirx.access.apprepo.api.config.authn.method.AuthnMethod"
        }
    ],
    "multiPepAssignments": [
        {
            "contextPath": "/odata4/config",
            "port": 11116
        }
    ],
    "cookie": {
        "name": "DXACookie"
    },
    "meta":{
        "patch":[
            {
                "key":"requestInjectionTemplateIds",
                "operation":{
                    "type":"remove"
                }
            },
            {
                "key":"allowedAuthnMethodIds",
                "operation":{
                    "type":"addItem"
                }
            },
            {
                "key":"multiPepAssignments",
                "operation":{
                    "type":"removeItem"
                }
            }
        ]
    }
}

Note that for the default operation replace (even inside an object) it is not needed to specify the replace operation in the patch metadata, as it is the default operation for properties without metadata.

Below is the result of the request, showing the state of the resource after the request is processed by the server.

{
    "@odata.context": "https://my-server.my-company.example:10116/odata4/config/1_0_0/$metadata#PlainWebPeps/$entity",
    "dp": "net.atos.dirx.access.apprepo.api.config.client.web.PlainWebPep",
    "key": "DirX Access PEP",
    "sslEnabled": true,
    "cacheTimeoutSecs": 1800,
    "clusterGroup": 0,
    "type": "Plain",
    "authority": "dxa-server:2",
    "nonPermitOrDenyMapping": "Deny",
    "xacmlRequestConstructionTemplateId": {
        "key": "Default Request Template",
        "type": "net.atos.dirx.access.apprepo.api.config.authz.XacmlRequestConstructionTemplate"
    },
    "xacmlPdpId": {
        "key": "RBAC Administration",
        "type": "net.atos.dirx.access.apprepo.api.config.authz.pdp.XacmlPdp"
    },
    "allowedAuthnMethodIds": [
        {
            "key": "Form Authentication",
            "type": "net.atos.dirx.access.apprepo.api.config.authn.method.AuthnMethod"
        },
        {
            "key": "Basic Authentication",
            "type": "net.atos.dirx.access.apprepo.api.config.authn.method.AuthnMethod"
        },
        {
            "key": "FIDO Authentication",
            "type": "net.atos.dirx.access.apprepo.api.config.authn.method.AuthnMethod"
        }
    ],
    "multiPepAssignments": [
        {
            "contextPath": "/odata4/config",
            "port": 11111,
            "pepId": {
                "key": "DirX Access PEP Internal clients",
                "type": "net.atos.dirx.access.apprepo.api.config.client.Pep"
            }
        },
        {
            "contextPath": "/odata4/config",
            "port": 11112,
            "pepId": {
                "key": "DirX Access PEP Internal clients",
                "type": "net.atos.dirx.access.apprepo.api.config.client.Pep"
            }
        },
        {
            "contextPath": "/odata4/sysactions",
            "port": 11116,
            "pepId": {
                "key": "DirX Access PEP OAuth RS",
                "type": "net.atos.dirx.access.apprepo.api.config.client.Pep"
            }
        },
        {
            "contextPath": "/scim",
            "port": 11116,
            "pepId": {
                "key": "DirX Access PEP OAuth RS",
                "type": "net.atos.dirx.access.apprepo.api.config.client.Pep"
            }
        }
    ],
    "cookie": {
        "name": "DXACookie",
        "path": "/",
        "version": 0,
        "secureEnabled": false,
        "httpOnly": false,
        "persistentCookieEnabled": false,
        "sameSiteFlag": "NotIncluded"
    }
}

Using patch metadata in startup config files

The process of using config files in the startup of DXA is described in the Direct Application of Configuration section.

The patch metadata can only be used for configuration objects that already exist in the system (modifying existing configuration objects). The syntax is the same for the patch metadata in the config files as for the PATCH requests, so it can be used to perform fine-grained patching of resources. The only difference is that the builder object in the config file must include the key and dp and @odata.type property, as it is used to identify the resource in the system.

Example of how a startup config file could look like, to modify the PlainWebPep to add a new allowed authentication method.

{
   "value":[
      {
         "key":"DirX Access PEP",
         "dp":"net.atos.dirx.access.apprepo.api.config.client.web.PlainWebPep",
         "@odata.type":"#net.atos.dirx.access.apprepo.api.config.client.web.PlainWebPep",
         "allowedAuthnMethodIds":[
            {
               "key":"FIDO Authentication",
               "type":"net.atos.dirx.access.apprepo.api.config.authn.method.AuthnMethod"
            }
         ],
         "meta":{
            "patch":[
               {
                  "key":"allowedAuthnMethodIds",
                  "operation":{
                     "type":"addItem"
                  }
               }
            ]
         }
      }
   ]
}