topic Re: Text Analytics concepts in SAS Data Science

Text Analytics concepts

TzufRaifMia — Sun, 02 Jul 2023 10:41:32 GMT

Hey,

I'm trying to formulizing the following condition into a concept in SAS Visual Text Analytics:

match a document if it found some word (defined in another concept) and it is not followed another concepts.

for example:

target: "pasta"

forbid: "eat"

"I eat pizza" -> not match

"I ___ pizza" -> when "____" is not "eat" -> match

Thanks in advance!

Re: Text Analytics concepts

sbxkoenk — Sun, 09 Jul 2023 10:48:51 GMT

Hello,

Your post was in

Home >> Welcome >> All Things Community

That's not the appropriate place for Data Science / Data Mining / Text Mining (analytics) / Machine Learning questions.

I have moved your post to
Home >> Analytics >> SAS Data Science
I dare to hope that being helped here (in the Data Science board) will be successful.

Good luck,

Koen

Re: Text Analytics concepts

toror5 — Wed, 09 Aug 2023 13:44:15 GMT

I'd say text analysis is hard, primarily because a machine learning model doesn't understand text, only numbers. Text analysis requires finding a way to represent the text and sentence structure with numbers in a way that keeps the usefulness of the information, such as making a TF-IDF matrix. If you're gauging popularity, you might want to look into libraries with functions for sentiment analysis, such as Bert.

Re: Text Analytics concepts

sbxkoenk — Wed, 09 Aug 2023 16:21:49 GMT

Hello @toror5 ,

Not sure why you are posting this.

With SAS Visual Text Analytics ( VTA ) you can fit machine learning models on top of unstructured data (textual data), but the original question was about Concept Rules for Information Extraction. That is a key element of natural language processing ( NLP ) and you do not necessarily need a model for it.

Concept rules are written using LITI (language interpretation and text interpretation) syntax.
Concept rules recognize items in context so that you can extract only the pieces of the document that match the rule.

[EDIT] I see now it was your 1st post as a new "member". Welcome to the SAS Communities!

BR, Koen

Re: Text Analytics concepts

Meilan — Thu, 10 Aug 2023 02:23:30 GMT

Hello,

You can first define a concept named 'Food' using the following rules:

# The words you want to match
CLASSIFIER:pasta
CLASSIFIER:pizza

# Remove the matches for the words that follow the verb eat. The 'eat@' represents all variations of the verb "eat".
REMOVE_ITEM:(ALIGNED, "eat@ _c{Food}", "eat@ Food")

Then you'll get the results like
I like pizza. -> match
I like pasta. -> match
I eat pizza. -> not match
I ate pasta. -> not match