BookmarkSubscribeRSS Feed
TzufRaifMia
Calcite | Level 5

Hey,

I'm trying to formulizing the following condition into a concept in SAS  Visual Text Analytics:

match a document if it found some word (defined in another concept) and it is not followed another concepts.

for example:

target: "pasta"

forbid: "eat"

 

 

"I eat pizza" -> not match

"I ___ pizza" -> when "____" is not "eat" -> match

 

Thanks in advance!

 

4 REPLIES 4
sbxkoenk
SAS Super FREQ

Hello,

 

Your post was in 

Home >> Welcome >> All Things Community

 

That's not the appropriate place for Data Science / Data Mining / Text Mining (analytics) / Machine Learning questions.

 

I have moved your post to 
Home >> Analytics >> SAS Data Science
I dare to hope that being helped here (in the Data Science board) will be successful.


Good luck,

Koen

toror5
Calcite | Level 5

I'd say text analysis is hard, primarily because a machine learning model doesn't understand text, only numbers. Text analysis requires finding a way to represent the text and sentence structure with numbers in a way that keeps the usefulness of the information, such as making a TF-IDF matrix. If you're gauging popularity, you might want to look into libraries with functions for sentiment analysis, such as Bert.

sbxkoenk
SAS Super FREQ

Hello @toror5 ,

 

Not sure why you are posting this.

 

With SAS Visual Text Analytics ( VTA ) you can fit machine learning models on top of unstructured data (textual data), but the original question was about Concept Rules for Information Extraction. That is a key element of natural language processing ( NLP ) and you do not necessarily need a model for it.

Concept rules are written using LITI (language interpretation and text interpretation) syntax.
Concept rules recognize items in context so that you can extract only the pieces of the document that match the rule.

 

[EDIT] I see now it was your 1st post as a new "member". Welcome to the SAS Communities!

 

BR, Koen

 
Meilan
SAS Employee

Hello,

 

You can first define a concept named 'Food' using the following rules:

# The words you want to match
CLASSIFIER:pasta
CLASSIFIER:pizza

# Remove the matches for the words that follow the verb eat. The 'eat@' represents all variations of the verb "eat".
REMOVE_ITEM:(ALIGNED, "eat@ _c{Food}", "eat@ Food")

 

Then you'll get the results like
I like pizza. -> match
I like pasta. -> match
I eat pizza. -> not match
I ate pasta. -> not match 

Ready to join fellow brilliant minds for the SAS Hackathon?

Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. Registration is now open through August 30th. Visit the SAS Hackathon homepage.

Register today!
How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 4 replies
  • 1769 views
  • 0 likes
  • 4 in conversation