I have to analyize a tweet for example :
This is not good at all.
we have a table which includes words with there polarity values.
eg
Word polarity
good +3
not good -3
When i pass this through code i should get a polarity of tweet as -3 and not +3.
"good" has a positive polarity but since a "not" comes before it the whole meaning changes.
Help me out with this one.
What's the ngram of your look up table?
I am not clear with what exactly ngram refers to.
# of words per phrase
good - 1-gram
not good 2-gram
really not good 3-gram
What is the max ngram of your lookup table?
Also, what's 'Advanced SAS'? Do you have EM with text analytics?
Advanced SAS using proc sql or macros. and No i don't have em with text analytics.
@Reeza wrote:
# of words per phrase
good - 1-gram
not good 2-gram
really not good 3-gram
What is the max ngram of your lookup table?
3
A data step is all you need.
1. Create a format for lookup table
2. Break a sentence into words. Since these are tweets you have a limit of words so splitting into an array should be easy enough. CountW will tell you how many words.
3a. Loop through each word for 1 gram, apply format and if found output score.
3b. Loop through again for 2 gram, apply format and if found output score
3c. 3a/3b for 3 gram.
ya i was able to get to the part of splitting words. bt i still didnt get how to apply the n gram logic.
It would be great if you can help me with a sample code for above example.
Sample data?
Here one is attached file containing 5 tweets and the other file has around 2400 words with their polarity scores.
I'm not downloading a zip file. Most won't either.
Add as plain text into your post, ideally as a data step
The question and code here are relevant, but the techniques may be beyond your programming skills. I'm a proponent of understanding your code.
Does your polarity table contains priority of checking words ?
Which columns do you have in this polarity table ? Can you add columns to it ?
Using a simple data step with indexw() function can do the work,
here, without using a polarity table, but just coding your example:
data _NULL_;
text = 'This is not good at all. ';
if indexw(text, 'not good') > 0 then polarity=-3; else /* priority 1 */
if indexw(text,'good') > 0 then polarity = 3; /* priority 2 */
put polarity=;
run;
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.