topic Re: Perform Sentimental Analysis Using Base +advanced Sas. in SAS Programming

Perform Sentiment Analysis Using Base +advanced Sas.

wizkid2050 — Sat, 22 Oct 2016 22:44:50 GMT

I have to analyize a tweet for example :

This is not good at all.

we have a table which includes words with there polarity values.

eg
Word polarity

good +3

not good -3

When i pass this through code i should get a polarity of tweet as -3 and not +3.

"good" has a positive polarity but since a "not" comes before it the whole meaning changes.

Help me out with this one.

Re: Perform Sentimental Analysis Using Base +advanced Sas.

Reeza — Sat, 22 Oct 2016 22:02:14 GMT

What's the ngram of your look up table?

Re: Perform Sentimental Analysis Using Base +advanced Sas.

wizkid2050 — Sat, 22 Oct 2016 22:09:31 GMT

I am not clear with what exactly ngram refers to.

Re: Perform Sentimental Analysis Using Base +advanced Sas.

Reeza — Sat, 22 Oct 2016 22:44:05 GMT

# of words per phrase

good - 1-gram

not good 2-gram

really not good 3-gram

What is the max ngram of your lookup table?

Re: Perform Sentimental Analysis Using Base +advanced Sas.

Reeza — Sat, 22 Oct 2016 22:45:28 GMT

Also, what's 'Advanced SAS'? Do you have EM with text analytics?

Re: Perform Sentimental Analysis Using Base +advanced Sas.

wizkid2050 — Sat, 22 Oct 2016 22:48:38 GMT

Advanced SAS using proc sql or macros. and No i don't have em with text analytics.

Re: Perform Sentimental Analysis Using Base +advanced Sas.

Reeza — Sat, 22 Oct 2016 22:59:21 GMT

@Reeza wrote:

# of words per phrase

good - 1-gram

not good 2-gram

really not good 3-gram

What is the max ngram of your lookup table?

Re: Perform Sentimental Analysis Using Base +advanced Sas.

wizkid2050 — Sat, 22 Oct 2016 23:08:16 GMT

Re: Perform Sentimental Analysis Using Base +advanced Sas.

Reeza — Sat, 22 Oct 2016 23:26:20 GMT

A data step is all you need.

1. Create a format for lookup table

2. Break a sentence into words. Since these are tweets you have a limit of words so splitting into an array should be easy enough. CountW will tell you how many words.

3a. Loop through each word for 1 gram, apply format and if found output score.

3b. Loop through again for 2 gram, apply format and if found output score

3c. 3a/3b for 3 gram.

Re: Perform Sentimental Analysis Using Base +advanced Sas.

wizkid2050 — Sat, 22 Oct 2016 23:32:13 GMT

ya i was able to get to the part of splitting words. bt i still didnt get how to apply the n gram logic.
It would be great if you can help me with a sample code for above example.

Re: Perform Sentimental Analysis Using Base +advanced Sas.

Reeza — Sat, 22 Oct 2016 23:56:56 GMT

Sample data?

Re: Perform Sentimental Analysis Using Base +advanced Sas.

Reeza — Sun, 23 Oct 2016 00:18:19 GMT

The question and code here are relevant, but the techniques may be beyond your programming skills. I'm a proponent of understanding your code.

https://communities.sas.com/t5/Base-SAS-Programming/Big-Data-Cartesian/m-p/288303/highlight/true#M59418

Re: Perform Sentimental Analysis Using Base +advanced Sas.

wizkid2050 — Sun, 23 Oct 2016 00:43:47 GMT

Here one is attached file containing 5 tweets and the other file has around 2400 words with their polarity scores.

Re: Perform Sentimental Analysis Using Base +advanced Sas.

Reeza — Sun, 23 Oct 2016 01:16:33 GMT

I'm not downloading a zip file. Most won't either.

Add as plain text into your post, ideally as a data step

https://communities.sas.com/t5/SAS-Communities-Library/How-to-create-a-data-step-version-of-your-data-AKA-generate/ta-p/258712

Re: Perform Sentiment Analysis Using Base +advanced Sas.

Shmuel — Sun, 23 Oct 2016 10:01:32 GMT

Does your polarity table contains priority of checking words ?

Which columns do you have in this polarity table ? Can you add columns to it ?

Using a simple data step with indexw() function can do the work,

here, without using a polarity table, but just coding your example:

data _NULL_;

text = 'This is not good at all. ';

if indexw(text, 'not good') > 0 then polarity=-3; else /* priority 1 */

if indexw(text,'good') > 0 then polarity = 3; /* priority 2 */

put polarity=;

run;