Custom Concept in Visual Text Analytics

[adupendse26]

Hello Experts,

I am trying to generate a model for Named Entity Recognition which would classify concepts (entities) from unstructured content using SAS Viya 3.4 Visual Text Analytics. 

I have tried Pre-defined concepts of Person and Organization as per requirement, however I observed that Pre-defined concepts are not properly trained and failed to classify few names and organization from my Content. How can I leverage custom concept to identify various person, organization from the unstructured data.

[adupendse26]

Hello Experts,

I am trying to generate a model for Named Entity Recognition which would classify concepts (entities) from unstructured content using SAS Viya 3.4 Visual Text Analytics and showcase as visualization in Visual Analytics

I have tried Pre-defined concepts of Person and Organization as per requirement, however I observed that Pre-defined concepts are not properly trained and failed to classify few names and organization from my Content. How can I leverage custom concept to identify various person, organization from the unstructured data

[TeresaJade]

Hi! So glad you asked about the predefined concepts! Let me start by saying that no NLP approach or model can work 100% especially across all types of data. However, when you use predefined concepts, they are intended as a good starting point for your analysis. The LITI language means that you can expand and grow that model to your needs.

You can add rules that result in more matches by using CLASSIFIER, CONCEPT, or C_CONCEPT, or CONCEPT_RULE type rules in the model. You can place them directly into the nlpOrganization concept, if you like, for expanding this concept. Or you can create your own myOrganization concept, and add your rules there under a rule CONCEPT:nlpOrganization.

You can also remove matches from the OOTB model by using the REMOVE_ITEM rule type. These rules are a bit harder to use, but a starter rule is here:

REMOVE_ITEM:(ALIGNED, "_c{nlpOrganization)", "bad match")

This rule says to remove the nlpOrganization match specified after the comma. Be sure that if you copy/past the rule, the quotes are not "smart quotes".

You can get more tips and tricks for using LITI rules and working with predefined concepts in the SAS Press book: https://support.sas.com/en/books/authors/teresa-jade.html. You can discuss ways to get this book with your SAS representative.