BookmarkSubscribeRSS Feed
supp
Pyrite | Level 9
I am attempting to extract topics from a collection of customer comments.

I would like to be able to parse common phrases.

I read the documentation for the multi term parameters within the parse statement. However I am not getting it to work.

How can I pass the phrases"I don't know", "improve training" or "everything is fine"?
4 REPLIES 4
sbxkoenk
SAS Super FREQ

It should work.

  • Are you using SAS 9.4 or SAS Viya?
  • Do you get ERROR or WARNING messages in the LOG?
  • Can you provide us with your code?

Here's an example of how that option should be used:

SAS® Visual Data Mining and Machine Learning Procedures
The TEXTMINE Procedure
Example 10.7 Adding a Multiterm List
https://go.documentation.sas.com/doc/en/vdmmlcdc/8.1/casml/viyaml_textmine_examples07.htm

 

Koen

supp
Pyrite | Level 9

Thanks for the reply!

 

I am using SAS Viya 3.5. I do not get any error or messages. My results are unchanged when I use multiterm. 

 

For example,  I put the term "dont know" in a CAS table "mycas._multiterm" as follows. Note that this is a common bi-gram in my data found by using pointwise mutual information.

supp_0-1716208155579.png

 

 

Here is my code for topic discovery:


proc textmine data= mycas.data_file;
	doc_id id;
	var text;
	parse
		termwgt= 	none
		cellwgt= 	none
		reducef= 	30
		entities= 	std
		multiterm= 	mycas._multiterm
		outparent= 	mycas.outparent
		outterms= 	mycas.outterms
		outpos=		mycas.outpos
		outchild=	mycas.outchild;
	svd	
		k=			100
		outdocpro=	mycas.outdocpro keepvariables= (id)
		outtopics=	mycas.outtopics
		svdu=  		mycas.outsvdu;
run;

 My primary indication that it is not working is that the term 'dont know' is not in mycas.outterms table. My secondary indication is that I can't get my topic assignments to change when I specify multi term phrases.

proc sql;
	select * from mycas.outterms where term = 'dont know';
quit;

supp_1-1716208594806.png

 

sbxkoenk
SAS Super FREQ

@supp wrote:

supp_0-1716208155579.png

Hmmmm.
Can you try

dont know :3:Prep

... instead of the way you entered it above?

Koen

supp
Pyrite | Level 9

The formatting you suggested might be the solution. I put a space after the term, then no other spaces.

 

I now see the term 'dont know' in my outterms CAS table!

supp_0-1716210376776.png

I will examine the results a bit more but this is progress!

Ready to join fellow brilliant minds for the SAS Hackathon?

Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. Registration is now open through August 30th. Visit the SAS Hackathon homepage.

Register today!
How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 4 replies
  • 428 views
  • 0 likes
  • 2 in conversation