BookmarkSubscribeRSS Feed

Using parameters and functions for multi-term search in Visual Analytics

Started ‎04-05-2023 by
Modified ‎04-12-2023 by
Views 691

A recent customer requirement to allow searching for data items using multiple terms captured using text box object in Visual Analytics had me thinking.

 

Understanding the Problem

 

To frame the problem it is important to understand the data representation for this particular use case. PDF documents are converted into structured datasets. Text analytics functions are performed on the data to identify specific key terms from the data. The terms matched in these documents are concatenated together with a separator (in this case single-pipe |) and stored in the datasets. The user wanted the ability to search for any combination of terms and the report objects would then filter the records containing all or any number of terms to display which documents contained the terms. A final ask form the customer was to use "out-of-the-box" Visual Analytics functionality and not use any custom objects or scripts or workarounds. 

 

Below is a small example of the dataset to better help understand the use case.

Gerrit_0-1680745249006.png

 

The following video showcases the requirement.

 

 

How to address the requirement

 

First we start by placing a Text object in the report page. Next a parameter (ENVN_EPA_HAB_cat_param) is created and assigned to the text object's parameter role. Note that "multiple value" parameters are not supported in this use case. 

 

Gerrit_0-1680748189071.png

 

Our text input object will allow the user to capture up to six terms separated by a comma. Note that using 6 terms in this scenario was by design and for your use cases you can define as many as you require following the same principles as described here. 

 

Next we have to parse the entries typed into the Text object, the value stored in the parameter linked to the text object. To do this we require six calculated items. Each calculated item will serve to parse the parameter (ENVN_EPA_HAB_cat_param) by using the "GetWord" function. The example below shows the custom function to get the 3rd word in the typed list of terms. 

 

Gerrit_1-1680748840670.png

 

The function GetWord follows the structure, input value, in this case the parameter (ENVN_EPA_HAB_cat_param) and the number of the word to be parsed, in this case the third word. The default word delimiter is comma. The six calculated items are named: EVEPHA_cat_1 through to EVEPHA_cat_6.

 

The second object we have to create on the report page is the List object and the column (keyTerms_EPA) which contains the terms we are searching against is mapped in the category role. We do this to display the result of the search after terms are input into the Text box above. This also allows this object to be linked to other objects in the report so that "Actions" can be mapped for interaction purposes. 

 

Gerrit_0-1680749628937.png

 

The secret sauce is the next step in the process. We now create and advanced filter for the List object. Below is the structure of the advanced filter.

 

Gerrit_1-1680749984819.png

 

From this advanced filter definition you can see the first step is to ensure that the parameter (ENVN_EPA_HAB_cat_param) contains text values. If it does, the expression continues by assessing if the term field in the dataset contains the subsequent terms 1 through to 6. Using the AND allows multiple terms to be searched against. If for example the custom function EVEPHA_cat_5 returns a null, it is ignored in the filter. The advanced filter subsequently allows the List object to be filtered with the values entered in the Text object as soon as the user completes entering terms in the Text object. 

 

Note however the process is only complete after the user clicks on the "Select all" option in the List object to ensure the List object recognizes the filtered items. All dependent objects linked to the List object will then be refreshed to reflect the filtered items.

 

Gerrit_0-1680751203431.png

 

Summary

 

Finding a solution to allowing a user to search through concatenated terms in a data field using a text box as term input was quite the challenge initially. There are most likely other approaches to solving this problem however after many tries this approach appears to function in a repeatable manner and one that provides the best performance. 

 

If you have an alternative view on potentially solving this problem in a different way please comment below.

 

 

 

 

 

 

 

Version history
Last update:
‎04-12-2023 12:41 AM
Updated by:
Contributors

SAS Innovate 2025: Call for Content

Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!

Submit your idea!

Free course: Data Literacy Essentials

Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning  and boost your career prospects.

Get Started

Article Labels
Article Tags