Automating indexing for Entities and Relashionships on SAS Visual Investigator

iakrim — Tue, 28 Jan 2020 08:11:29 GMT

Hello Community,

Am looking for a way to automate the process of indexing entities and relashionships upon updating the data behind the scenes.

I found no way to create a job/Schedule it regarding data indexing in SAS Visual Investiator.

Re: Automating indexing for Entities and Relashionships on SAS Visual Investigator

Rory_MacKenzie — Tue, 28 Jan 2020 11:47:28 GMT

Hi Iakrim,

There are instructions on how to call our indexing APIs in the Visual Investigator Admin Help documentation. If you open the documentation then search for the following section - "Re-Indexing, Resolving Entities, and Running Flows Programmatically" - then this should provide you the details you need.

Thanks,

Rory

Re: Automating indexing for Entities and Relashionships on SAS Visual Investigator

iakrim — Wed, 19 Feb 2020 15:35:53 GMT

Step by step solution :

(localhost can be adapted to the microservices IP)

I . This uses an API so you should have an app name and get a token any time for that :

to do for the first time ever (by client app)

1 - Get the token from consul on the microservices server :

cat /opt/sas/viya/config/etc/SASSecurityCertificateFramework/tokens/consul/default/client.token

Result is a string1 : XXXXXX-XXXXX-XXXX-XXXXXX-XXXXX

2 - Declare a new client as en example NWCLI and copy the string1 into the command:

curl -k -X POST "https://localhost/SASLogon/oauth/clients/consul?callback=false&serviceId=NWCLI" -H "X-Consul-Token: XXXXXX-XXXXX-XXXX-XXXXXX-XXXXX"

Result String2 :
{"access_token":"XXXXXXXXXX....","token_type":"bearer","expires_in":35999,"scope........."}

we are only interested by the access token value.

3 - Register le new client with that token (localhost can be adapted to the microservices IP) :

curl -k -X POST https://localhost/SASLogon/oauth/clients -H "Content-Type: application/json" -H "Authorization: Bearer XXXXXX...." -d '{ "client_id": "NWCLI", "client_secret": "client-secret", "scope": ["openid", "*"], "resource_ids": "none", "authorities": ["uaa.none"], "authorized_grant_types": ["password"] }'

II . Since the client is already declared to use the API, by default the tokens expire in 12 hours. You can either expand the timeout or use a shell workaround :

1 - Getting the 12 hours token :

curl http://localhost/SASLogon/oauth/token -H "Accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" -d "grant_type=password&username=username&password=adminpassword" -u "NWCLI:client-secret"

Result is string3 : XXXXXXXXXXXXXXX...... to use in next step

2 - Send the index all in curl :

curl -k -d '{"parameters": { "type": "SEARCH_INDEX_LOADER" }}' -H "Content-Type: application/json" -H "Authorization: Bearer  XXXXXXXXXXXXXXX........" -X POST https://localhost/svi-datahub/admin/asyncJobs?new

{"parameters": { "type": "SEARCH_INDEX_LOADER" }} is for indexing all entities

Workaround to bypass the token timeout :
Create a .sh script to launch the II.1 and II.2 steps automatically :

var=$(curl http://localhost/SASLogon/oauth/token -H "Accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" -d "grant_type=password&username=...&password=..." -u "NWCLI:client-secret")
var1=$(echo $var | cut -c18-1326)
curl -k -d '{"parameters": { "type": "SEARCH_INDEX_LOADER" }}' -H "Content-Type: application/json" -H "Authorization: Bearer $var1" -X POST https://localhost/svi-datahub/admin/asyncJobs?new

You can CRON the script for data that refreshes regularly.

topic Automating indexing for Entities and Relashionships on SAS Visual Investigator in Fraud, AML and Security Intelligence

Automating indexing for Entities and Relashionships on SAS Visual Investigator

Re: Automating indexing for Entities and Relashionships on SAS Visual Investigator

Re: Automating indexing for Entities and Relashionships on SAS Visual Investigator