Automating indexing for Entities and Relashionships on SAS Visual Investigator

[iakrim]

Hello Community, 

Am looking for a way to automate the process of indexing entities and relashionships upon updating the data behind the scenes. 

I found no way to create a job/Schedule it regarding data indexing in SAS Visual Investiator.

[Rory_MacKenzie · Accepted solution]

Hi Iakrim,

There are instructions on how to call our indexing APIs in the Visual Investigator Admin Help documentation. If you open the documentation then search for the following section - "Re-Indexing, Resolving Entities, and Running Flows Programmatically" - then this should provide you the details you need. 

Thanks,

Rory

[iakrim]

Step by step solution : 

(localhost can be adapted to the microservices IP) 

I . This uses an API so you should have an app name and get a token any time for that :

to do for the first time ever (by client app)  

1 - Get the token from consul on the microservices server :
 

cat /opt/sas/viya/config/etc/SASSecurityCertificateFramework/tokens/consul/default/client.token

Result is a string1 : XXXXXX-XXXXX-XXXX-XXXXXX-XXXXX

2 - Declare a new client as en example  NWCLI and copy the string1 into the command:

curl -k -X POST "https://localhost/SASLogon/oauth/clients/consul?callback=false&serviceId=NWCLI" -H "X-Consul-Token: XXXXXX-XXXXX-XXXX-XXXXXX-XXXXX"

Result String2 :
{"access_token":"XXXXXXXXXX....","token_type":"bearer","expires_in":35999,"scope........."}

we are only interested by the access token value.

3 - Register le new client with that token  (localhost can be adapted to the microservices IP) :

curl -k -X POST https://localhost/SASLogon/oauth/clients -H "Content-Type: application/json" -H "Authorization: Bearer XXXXXX...." -d '{ "client_id": "NWCLI", "client_secret": "client-secret", "scope": ["openid", "*"], "resource_ids": "none", "authorities": ["uaa.none"], "authorized_grant_types": ["password"] }'

II . Since the client is already declared to use the API, by default the tokens expire in 12 hours. You can either expand the timeout or use a shell workaround : 

1 - Getting the 12 hours token  :

curl http://localhost/SASLogon/oauth/token -H "Accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" -d "grant_type=password&username=username&password=adminpassword" -u "NWCLI:client-secret"

Result  is string3 : XXXXXXXXXXXXXXX...... to use in next step 

2 - Send the index all in curl :

curl -k -d '{"parameters": { "type": "SEARCH_INDEX_LOADER" }}' -H "Content-Type: application/json" -H "Authorization: Bearer  XXXXXXXXXXXXXXX........" -X POST https://localhost/svi-datahub/admin/asyncJobs?new

{"parameters": { "type": "SEARCH_INDEX_LOADER" }} is for indexing all entities

Workaround to bypass the token timeout : 
Create a .sh script to launch the II.1 and II.2 steps automatically : 

var=$(curl http://localhost/SASLogon/oauth/token -H "Accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" -d "grant_type=password&username=...&password=..." -u "NWCLI:client-secret")
var1=$(echo $var | cut -c18-1326)
curl -k -d '{"parameters": { "type": "SEARCH_INDEX_LOADER" }}' -H "Content-Type: application/json" -H "Authorization: Bearer $var1" -X POST https://localhost/svi-datahub/admin/asyncJobs?new

You can CRON the script for data that refreshes regularly.