BookmarkSubscribeRSS Feed
wizkid2050
Fluorite | Level 6

I have to analyize a tweet for example : 

This is not good at all. 

 

we have a table which includes words with there polarity values.

eg 
Word     polarity 

good       +3

not good  -3

 

When i pass this through code i should get a polarity of tweet as -3 and not +3.

"good" has a positive polarity but since a "not" comes before it the whole  meaning changes.

Help me out with this one.

 

 

 

 

14 REPLIES 14
Reeza
Super User

What's the ngram of your look up table? 

wizkid2050
Fluorite | Level 6

I am not clear with what exactly ngram refers to.

Reeza
Super User

# of words per phrase 

 

good - 1-gram

not good 2-gram

really not good 3-gram

 

What is the max ngram of your lookup table?

Reeza
Super User

Also, what's 'Advanced SAS'? Do you have EM with text analytics? 

wizkid2050
Fluorite | Level 6

Advanced SAS using proc sql or macros. and No i don't have em with text analytics.

Reeza
Super User

@Reeza wrote:

# of words per phrase 

 

good - 1-gram

not good 2-gram

really not good 3-gram

 

What is the max ngram of your lookup table?




 

Reeza
Super User

A data step is all you need. 

 

1. Create a format for lookup table

2. Break a sentence into words. Since these are tweets you have a limit of words so splitting into an array should be easy enough. CountW will tell you how many words. 

3a. Loop through each word for 1 gram, apply format and if found output score. 

3b. Loop through again for 2 gram, apply format and if found output score

3c. 3a/3b for 3 gram. 

 

 

wizkid2050
Fluorite | Level 6

ya i was able to get to the part of splitting words. bt i still didnt get how to apply the n gram logic. 
It would be great if you can help me with a sample code for above example.

wizkid2050
Fluorite | Level 6

Here one is attached file containing 5 tweets and the other file has around 2400 words with their polarity scores.

Reeza
Super User

I'm not downloading a zip file. Most won't either. 

Add as plain text into your post, ideally as a data step

 

https://communities.sas.com/t5/SAS-Communities-Library/How-to-create-a-data-step-version-of-your-dat...

Reeza
Super User

The question and code here are relevant, but the techniques may be beyond your programming skills. I'm a proponent of understanding your code. 

 

https://communities.sas.com/t5/Base-SAS-Programming/Big-Data-Cartesian/m-p/288303/highlight/true#M59...

Shmuel
Garnet | Level 18

Does your polarity table contains priority of checking words ?

Which columns do you have in this polarity table ? Can you add columns to it ?

 

Using a simple data step with indexw() function can do the work,

here, without using a polarity table, but just coding your example:

 

data _NULL_;

     text = 'This is not good at all. ';

     if indexw(text, 'not good') > 0 then polarity=-3; else  /* priority 1 */

     if indexw(text,'good') > 0 then polarity = 3;                /* priority 2 */

     put polarity=;

run;

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 14 replies
  • 2028 views
  • 0 likes
  • 3 in conversation