vi entity index and inmemory load automatically

[yjchoi]

Platform : sas visual investigation
When the original data of an external entity used in vi is updated or added with sas lsf, should the user do inmemory load and indexing in vi? Is there a way to do it automatically?

[JeanDo]

Hello,

I'm not familiar with SAS LSF but if you want to update your external entity and see the changes into VI, you just have to reindex the entity after updating your table. You can do it automatically with rest api calls; in SAS you should use the proc http: you just have to change the fields between "<>":

# First you have to get the authentication token 
filename resp TEMP;
filename hdrs TEMP;
proc http url="http://<your-domain-name>/SASLogon/oauth/token" 
			method='POST' auth_basic webusername="sas.ec" webpassword=""
			in='grant_type=password&username=<your-user-name>&password=<your-password>'
			out=resp
			headerout=hdrs;
run;

# Put the token in the access_token macro variable
data _null_;
infile resp truncover dlm='"' ;
input @'"access_token":"' token :$2000. ;
call symputx('access_token',token,'g');
run;


# Launch the reindexing job through the API
proc http url="http://<your-domain-name>/svi-datahub/admin/asyncJobs?new"
method='POST' OAUTH_BEARER="&access_token."
in='{"parameters": {"type": "SEARCH_INDEX_LOADER"},"tasks": [{"parameters": {"type": "LOAD_DOCUMENT","name": "<your-entity-name>"}}]}';
headers 'Accept'='application/json' 'Content-Type'='text/plain';
run;

Hope this will help.

Cheers,

JD 

[yjchoi]

Hello,
I confirmed your help late. Thank you for answer.
Your answer has been tested and re-indexed. We confirmed that data reflection during testing is possible with re-load. Don't know api well, do you also know re-load?
Once again thank you for your help.

[JeanDo]

Hello,

I'm happy to help; if I understand well the reindexing is OK for you, but you want to reload your data in-memory? 

Could you please describe what should be your reload process? Do you want to load data into a CAS table? Could you also tell me which type of datasource are you using for your external entity (PostgreSQL, MySQL, ...)?

Thank you for your answers,

JD

[yjchoi]

Hello,
Right now in vi the user manually re-loads the imported entities, but we want to do it with sas code.
We used external entity on postgresql
Also, the method you taught is only possible for one entity. Is there a way for all entities?
Thanks you.

[JeanDo]

Hi,
I'm still not sure to fully understand your point; what do you mean by "re-load"?
- when you said "the user manually re-loads the imported entities" I understand "the user manually launch the reindex of the imported entities". If it is also your point, I already showed you how to do it (see my code above). Or maybe the solution I gave you does not fit?
- or I am wrong in my understanding, and when you're talking about "re-load" you want to say that the user re-import the entities from scratch?

Depending on your response to the first point, I will be able to show you how to do it for all entities.

Have a nice day,

JD

[yjchoi]

Hello, JeanDO

We have tested the problem to clarify and communicate it to you. Don't worry about "Re-load". Only one entity can be indexed with the method you taught. We want all entities (tables) to be indexable.
Thanks you.

[JeanDo · Accepted solution]

Hello,

Thank you for the clarifications. To reindex all entities, please submit the following code:

# First you have to get the authentication token 
filename resp TEMP;
filename hdrs TEMP;
proc http url="http://<your-domain-name>/SASLogon/oauth/token" 
			method='POST' auth_basic webusername="sas.ec" webpassword=""
			in='grant_type=password&username=<your-user-name>&password=<your-password>'
			out=resp
			headerout=hdrs;
run;

# Put the token in the access_token macro variable
data _null_;
infile resp truncover dlm='"' ;
input @'"access_token":"' token :$2000. ;
call symputx('access_token',token,'g');
run;


# Launch the reindexing job through the API
proc http url="http://<your-domain-name>/svi-datahub/admin/asyncJobs?new"
method='POST' OAUTH_BEARER="&access_token."
in='{"parameters": {"type": "SEARCH_INDEX_LOADER"}}';
headers 'Accept'='application/json' 'Content-Type'='text/plain';
run;

I only changed the final request's input. Please let me know if you have additional questions.

Have a good day,

JD

[yjchoi]

Thanks JeanDO.

[avinashjha1787]

Hi

Is there any way to capture when the indexing is complete ?

I have jobs to be executed in batch post indexing. If I am able to capture when the indexing triggered via API call is complete, it would be helpful to trigger next job. Currently I am just using sleep function to hold the next job.