How to Delete a List of Entity Types Using Bash and Curl for SAS Visual Investigator

[#1]

As a SAS Visual Investigator user, you might need to delete a large number of entity types. This can be cumbersome using the web UI and is not supported using SQL due to data integrity issues-- what to do? This article shows you how to create a bash script that removes a list of entity types from SAS Visual Investigator. 

Caution: 

Deleting an entity type is a potentially destructive action. When an internal entity type is removed, the underlying data table and its associated records are also removed. This action can delete a significant amount of data and should be used with caution. All records of a given type will be removed.
For external entity types, this is less of a concern because only the reference to the external database table is removed. Visual Investigator does not make changes or remove data from external tables.

Running the Bash Script

To delete entity types, use a text file as an input parameter that provides the names of the types to be removed.

1. Add the bash script (removeTypes.sh) and the text file that contains the entity type names to the same directory. In the following example, the list of type names is stored in a file named typeList.txt:
   
2. The format for typeList.txt is as follows:

   first_entity_type
   second_entity_type
   third_entity_type

3. Before running the bash command, complete the following steps:
   • Provide access rights to the bash script.
   • Run the following command:

     chmod +x removeTypes.sh

4. Use a bash command to run the bash script from the folder that contains the script. To run the script, use the following command:

   $ ./removeTypes.sh http://acme.myserver.company.com MyUserId MyPassword typeList.txt

   Use the following parameters:

   • The URL of the target system.

   • The username of system.

   • The password.

   • The name of the text file that contains the list of entity types to delete.

How to Use the Bash Script

The bash script removeTypes.sh consists of different bash commands. To begin, an access token must be generated, then the entities are deleted. 

1. Generate an access token using a curl command. The request is sent to the SASLogon service and the token is parsed from the response and stored in a variable named “access_token.”
   

   response=$(curl --request POST --url ${token_endpoint}/SASLogon/oauth/token --header 'accept: application/json' --header 'authorization: Basic c2FzLmVjOg==' --header 'content-type: application/x-www-form-urlencoded' --data grant_type=password --data username=${username} --data password=${password})
   
   access_token=$(echo "$response" | sed 's/[{}"]//g' | awk -F, '{for (i=1; i<=NF; i++) {if ($i ~ /access_token/) {print $i}}}' | awk -F: '{print $2}')

2. Use a for loop to delete he entities from three microservices: Datahub, SAND and Feature. You can create three URLs named as delete URLs.
   The curl command to delete from Datahub is as follows:

    # Send DELETE request to datahub microservice
   
   response_datahub=$(curl -X DELETE "${delete_url}?name=${entity_name}" \
   
   -H "Accept: application/json" \
   
   -H "Authorization: Bearer ${access_token}" \
   
   -w "\nResponse Code: %{http_code}\n")

   The curl command to delete from SAND is as follows:

    # Send DELETE request to SAND microservice
   
   response_sand=$(curl -X DELETE "${delete_url_sand}/${entity_name}" \
   
   -H "Accept: application/json" \
   
   -H "Authorization: Bearer ${access_token}" \
   
   -w "\nResponse Code: %{http_code}\n")

   The curl command to delete from the feature microservice is as follows:

    # Send DELETE request to feature microservice
   
   response_feature=$(curl -X DELETE "${delete_url_feature}/${entity_name}" \
   
   -H "Accept: application/json" \
   
   -H "Authorization: Bearer ${access_token}" \
   
   -w "\nResponse Code: %{http_code}\n")
   
   Once you run the bash script the entities will be deleted from above microservices.
   
   If you are accessing the UI after running bash script, please refresh the UI so that the recent changes will be acknowledged.

You have now removed a list of entity types from SAS Visual Investigator, thanks for using this tutorial.