BookmarkSubscribeRSS Feed
supp
Pyrite | Level 9
I am attempting to extract topics from a collection of customer comments.

I would like to be able to parse common phrases.

I read the documentation for the multi term parameters within the parse statement. However I am not getting it to work.

How can I pass the phrases"I don't know", "improve training" or "everything is fine"?
4 REPLIES 4
sbxkoenk
SAS Super FREQ

It should work.

  • Are you using SAS 9.4 or SAS Viya?
  • Do you get ERROR or WARNING messages in the LOG?
  • Can you provide us with your code?

Here's an example of how that option should be used:

SAS® Visual Data Mining and Machine Learning Procedures
The TEXTMINE Procedure
Example 10.7 Adding a Multiterm List
https://go.documentation.sas.com/doc/en/vdmmlcdc/8.1/casml/viyaml_textmine_examples07.htm

 

Koen

supp
Pyrite | Level 9

Thanks for the reply!

 

I am using SAS Viya 3.5. I do not get any error or messages. My results are unchanged when I use multiterm. 

 

For example,  I put the term "dont know" in a CAS table "mycas._multiterm" as follows. Note that this is a common bi-gram in my data found by using pointwise mutual information.

supp_0-1716208155579.png

 

 

Here is my code for topic discovery:


proc textmine data= mycas.data_file;
	doc_id id;
	var text;
	parse
		termwgt= 	none
		cellwgt= 	none
		reducef= 	30
		entities= 	std
		multiterm= 	mycas._multiterm
		outparent= 	mycas.outparent
		outterms= 	mycas.outterms
		outpos=		mycas.outpos
		outchild=	mycas.outchild;
	svd	
		k=			100
		outdocpro=	mycas.outdocpro keepvariables= (id)
		outtopics=	mycas.outtopics
		svdu=  		mycas.outsvdu;
run;

 My primary indication that it is not working is that the term 'dont know' is not in mycas.outterms table. My secondary indication is that I can't get my topic assignments to change when I specify multi term phrases.

proc sql;
	select * from mycas.outterms where term = 'dont know';
quit;

supp_1-1716208594806.png

 

sbxkoenk
SAS Super FREQ

@supp wrote:

supp_0-1716208155579.png

Hmmmm.
Can you try

dont know :3:Prep

... instead of the way you entered it above?

Koen

supp
Pyrite | Level 9

The formatting you suggested might be the solution. I put a space after the term, then no other spaces.

 

I now see the term 'dont know' in my outterms CAS table!

supp_0-1716210376776.png

I will examine the results a bit more but this is progress!

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 4 replies
  • 382 views
  • 0 likes
  • 2 in conversation