Text Analytics concepts

[TzufRaifMia]

Hey,

I'm trying to formulizing the following condition into a concept in SAS  Visual Text Analytics:

match a document if it found some word (defined in another concept) and it is not followed another concepts.

for example:

target: "pasta"

forbid: "eat"

"I eat pizza" -> not match

"I ___ pizza" -> when "____" is not "eat" -> match

Thanks in advance!

[sbxkoenk]

Hello,

Your post was in 

Home >> Welcome >> All Things Community

That's not the appropriate place for Data Science / Data Mining / Text Mining (analytics) / Machine Learning questions.

I have moved your post to 
Home >> Analytics >> SAS Data Science
I dare to hope that being helped here (in the Data Science board) will be successful.

Good luck,

Koen

[toror5]

I'd say text analysis is hard, primarily because a machine learning model doesn't understand text, only numbers. Text analysis requires finding a way to represent the text and sentence structure with numbers in a way that keeps the usefulness of the information, such as making a TF-IDF matrix. If you're gauging popularity, you might want to look into libraries with functions for sentiment analysis, such as Bert.

[sbxkoenk]

Hello @toror5 ,

Not sure why you are posting this.

With SAS Visual Text Analytics ( VTA ) you can fit machine learning models on top of unstructured data (textual data), but the original question was about Concept Rules for Information Extraction. That is a key element of natural language processing ( NLP ) and you do not necessarily need a model for it.

Concept rules are written using LITI (language interpretation and text interpretation) syntax.
Concept rules recognize items in context so that you can extract only the pieces of the document that match the rule.

[EDIT] I see now it was your 1st post as a new "member". Welcome to the SAS Communities!

BR, Koen

[Meilan]

Hello,

You can first define a concept named 'Food' using the following rules:

# The words you want to match
CLASSIFIER:pasta
CLASSIFIER:pizza

# Remove the matches for the words that follow the verb eat. The 'eat@' represents all variations of the verb "eat".
REMOVE_ITEM:(ALIGNED, "eat@ _c{Food}", "eat@ Food")

Then you'll get the results like
I like pizza. -> match
I like pasta. -> match
I eat pizza. -> not match
I ate pasta. -> not match