BookmarkSubscribeRSS Feed
wizkid2050
Fluorite | Level 6

I have to analyize a tweet for example : 

This is not good at all. 

 

we have a table which includes words with there polarity values.

eg 
Word     polarity 

good       +3

not good  -3

 

When i pass this through code i should get a polarity of tweet as -3 and not +3.

"good" has a positive polarity but since a "not" comes before it the whole  meaning changes.

Help me out with this one.

 

 

 

 

14 REPLIES 14
Reeza
Super User

What's the ngram of your look up table? 

wizkid2050
Fluorite | Level 6

I am not clear with what exactly ngram refers to.

Reeza
Super User

# of words per phrase 

 

good - 1-gram

not good 2-gram

really not good 3-gram

 

What is the max ngram of your lookup table?

Reeza
Super User

Also, what's 'Advanced SAS'? Do you have EM with text analytics? 

wizkid2050
Fluorite | Level 6

Advanced SAS using proc sql or macros. and No i don't have em with text analytics.

Reeza
Super User

@Reeza wrote:

# of words per phrase 

 

good - 1-gram

not good 2-gram

really not good 3-gram

 

What is the max ngram of your lookup table?




 

Reeza
Super User

A data step is all you need. 

 

1. Create a format for lookup table

2. Break a sentence into words. Since these are tweets you have a limit of words so splitting into an array should be easy enough. CountW will tell you how many words. 

3a. Loop through each word for 1 gram, apply format and if found output score. 

3b. Loop through again for 2 gram, apply format and if found output score

3c. 3a/3b for 3 gram. 

 

 

wizkid2050
Fluorite | Level 6

ya i was able to get to the part of splitting words. bt i still didnt get how to apply the n gram logic. 
It would be great if you can help me with a sample code for above example.

wizkid2050
Fluorite | Level 6

Here one is attached file containing 5 tweets and the other file has around 2400 words with their polarity scores.

Reeza
Super User

I'm not downloading a zip file. Most won't either. 

Add as plain text into your post, ideally as a data step

 

https://communities.sas.com/t5/SAS-Communities-Library/How-to-create-a-data-step-version-of-your-dat...

Reeza
Super User

The question and code here are relevant, but the techniques may be beyond your programming skills. I'm a proponent of understanding your code. 

 

https://communities.sas.com/t5/Base-SAS-Programming/Big-Data-Cartesian/m-p/288303/highlight/true#M59...

Shmuel
Garnet | Level 18

Does your polarity table contains priority of checking words ?

Which columns do you have in this polarity table ? Can you add columns to it ?

 

Using a simple data step with indexw() function can do the work,

here, without using a polarity table, but just coding your example:

 

data _NULL_;

     text = 'This is not good at all. ';

     if indexw(text, 'not good') > 0 then polarity=-3; else  /* priority 1 */

     if indexw(text,'good') > 0 then polarity = 3;                /* priority 2 */

     put polarity=;

run;

hackathon24-white-horiz.png

2025 SAS Hackathon: There is still time!

Good news: We've extended SAS Hackathon registration until Sept. 12, so you still have time to be part of our biggest event yet – our five-year anniversary!

Register Now

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

SAS Training: Just a Click Away

 Ready to level-up your skills? Choose your own adventure.

Browse our catalog!

Discussion stats
  • 14 replies
  • 4100 views
  • 0 likes
  • 3 in conversation