This article is part three of a three-part series on the new data masking capabilities in SAS Visual Investigator LTS 2023.10. In the previous article we discussed how the Datahub service handles masked values when processing PUT requests to update a document object. In this article, we'll learn more about the Datahub REST API's new support for PATCH requests.
Using PATCH to update a document object
The PUT method is only one way to update objects in the Datahub REST API. As of LTS 2023.10, the Datahub REST API supports PATCH requests to update document objects.
While the PUT method is for specifying a whole new representation to persist at some URL (in other words, a full update), the PATCH method is for specifying a partial update to the existing persisted representation. One advantage is that you don't have to send the whole document object to Datahub in order to update one value.
There are different ways to describe a set of changes you want to make to an object. The Datahub PATCH endpoints use the JSON Patch representation. Explaining JSON Patch is beyond the scope of this article, but you can read more about the specification at the JSON Patch website.
The Datahub PATCH endpoints do not allow certain JSON Patch operations to be applied to masked field values. Specifically, test, move, and copy are prohibited on masked field values. This prevents masked data from leaking via the results of these operations.
For Datahub's document PATCH endpoint, most updates will require the client to send an object version number in the If-Match header. (The exception is when the entity type does not have a field configured for the "version" role.) The If-Match header must contain the latest object version number for the document. This maintains data integrity by preventing potentially conflicting updates.
To find the version number, we must fetch the object from the Datahub service:
curl $VI_BASE/svi-datahub/documents/person/2f21e644-089a-47d8-a503-bbdd4d8dac3d -H "Authorization: Bearer $TOKEN" -H "Accept: application/json" | jq
Since the legacy representation and the masked representation both contain the version field, it doesn't matter which one we request here. In this example, the request is for the legacy representation.
For the Person entity type, the field version is configured to have the "version" role. So we can see from the response that the latest object version for this Person instance is 4:
{
"objectTypeName": "person",
"objectTypeId": 100515,
"objectTypeVersion": 4,
"id": "2f21e644-089a-47d8-a503-bbdd4d8dac3d",
"fieldValues": {
"birthday": "2020-01-05T00:00:00Z",
"created_at_dttm": "2020-04-13T19:17:47.84Z",
"created_by_user_id": "viuser",
"first_name": "John",
"id": "2f21e644-089a-47d8-a503-bbdd4d8dac3d",
"last_name": "Smith",
"last_updated_at_dttm": "2020-04-13T19:31:37.097Z",
"last_updated_by_user_id": "viuser",
"version": 4 // <- Here is the object version number
},
"createdAt": "2020-04-13T19:17:47.840Z",
"lastUpdatedAt": "2020-04-13T19:31:37.097Z",
"validFrom": "2020-01-05T00:00:00.000Z"
}
Now that we know what version we are working with, we can construct a JSON Patch to send in the PATCH request:
[
{
"op": "replace",
"path": "/fieldValues/ssn",
"value": "999-88-7777"
},
{
"op": "replace",
"path": "/fieldValues/first_name",
"value": "James"
}
]
We'll save this JSON Patch into a file called my-patch.json.
Like updating via PUT, updating via PATCH requires locking the document first:
curl "$VI_BASE/svi-datahub/locks/documents?type=person&id=2f21e644-089a-47d8-a503-bbdd4d8dac3d" -X POST -H "Authorization: Bearer $TOKEN"
Let's use curl to send the patch to the PATCH endpoint, remembering to set the If-Match header. We'll also set the Accept header to the masked media type so that we get back a masked representation.
curl $VI_BASE/svi-datahub/documents/person/2f21e644-089a-47d8-a503-bbdd4d8dac3d -X PATCH --upload-file my-patch.json -H "Authorization: Bearer $TOKEN" -H "If-Match: 4" -H "Content-Type: application/json-patch+json" -H "Accept: application/vnd.sas.investigation.data.masked.enriched.document+json" | jq
In the response, we can see that the value of the first_name field has been updated. Since the ssn field is masked, we can't see that it changed, however.
{
"objectTypeName": "person",
"objectTypeId": 100515,
"objectTypeVersion": 4,
"id": "2f21e644-089a-47d8-a503-bbdd4d8dac3d",
"fieldValues": {
"birthday": "2020-01-05T00:00:00Z",
"created_at_dttm": "2020-04-13T19:17:47.84Z",
"created_by_user_id": "viuser",
"first_name": "James",
"id": "2f21e644-089a-47d8-a503-bbdd4d8dac3d",
"last_name": "Smith",
"ssn": "•••••••••",
"pin": "•••••••••",
"last_updated_at_dttm": "2024-01-09T17:01:11.567Z",
"last_updated_by_user_id": "viuser",
"version": 5
},
"createdAt": "2020-04-13T19:17:47.840Z",
"lastUpdatedAt": "2024-01-09T17:01:11.567Z",
"validFrom": "2020-01-05T00:00:00.000Z",
"fieldRestrictions": {
"person": {
"ssn": {
"masked": {
"currentUserIsAuthorizedToReveal": true
}
},
"pin": {
"masked": {
"currentUserIsAuthorizedToReveal": false
}
}
}
}
}
In order to verify that the ssn field was updated, we can request the unmasked value of the field using curl:
curl $VI_BASE/svi-datahub/documents/person/2f21e644-089a-47d8-a503-bbdd4d8dac3d/fields/ssn -H "Authorization: Bearer $TOKEN" | jq
In the response we see that the value is 999-88-7777, as we expect:
{
"raw": "999-88-7777"
}
The JSON Patch specification allows for much more powerful and complex operations to be performed. However, when working with masked fields, be aware that test, move, and copy operations cannot be applied to masked field values, as this would circumvent Datahub's authorization mechanism for revealing those fields' values.
Conclusion
This series of blog posts has explained the data model and media types that API consumers use to work with masked fields within the Datahub service. It has also shown several examples of working with masked data using curl and the Datahub REST API. All of this knowledge can be applied to build powerful automated consumers of the Datahub REST API, or to update existing consumers to be aware of masked data.
Further Reading
For more details about masked representations and media types, refer to the examples page in the Datahub REST API documentation. For more details about GET, PUT and PATCH operations on document objects, refer to the Documents section of the Datahub REST API documentation.
In LTS 2023.10, SAS Visual Investigator also introduced support for data masking of relationship (link) fields. The principles I discussed in this article also apply to masked relationship fields. For more details on masked representations for relationship fields, refer to the examples page of the Datahub REST API documentation. For details about how masked data is handled in relationship GET, PUT, and PATCH operations, refer to the Relationship Links section of the Datahub REST API documentation.
