topic Re: Specify multi word phrases in proc textmine in SAS Data Science

Specify multi word phrases in proc textmine

supp — Mon, 20 May 2024 02:05:06 GMT

I am attempting to extract topics from a collection of customer comments.

I would like to be able to parse common phrases.

I read the documentation for the multi term parameters within the parse statement. However I am not getting it to work.

How can I pass the phrases"I don't know", "improve training" or "everything is fine"?

Re: Specify multi word phrases in proc textmine

sbxkoenk — Mon, 20 May 2024 12:03:45 GMT

It should work.

Are you using SAS 9.4 or SAS Viya?
Do you get ERROR or WARNING messages in the LOG?
Can you provide us with your code?

Here's an example of how that option should be used:

SAS® Visual Data Mining and Machine Learning Procedures
The TEXTMINE Procedure
Example 10.7 Adding a Multiterm List
https://go.documentation.sas.com/doc/en/vdmmlcdc/8.1/casml/viyaml_textmine_examples07.htm

Koen

Re: Specify multi word phrases in proc textmine

supp — Mon, 20 May 2024 12:43:45 GMT

Thanks for the reply!

I am using SAS Viya 3.5. I do not get any error or messages. My results are unchanged when I use multiterm.

For example, I put the term "dont know" in a CAS table "mycas._multiterm" as follows. Note that this is a common bi-gram in my data found by using pointwise mutual information.

Here is my code for topic discovery:


proc textmine data= mycas.data_file;
	doc_id id;
	var text;
	parse
		termwgt= 	none
		cellwgt= 	none
		reducef= 	30
		entities= 	std
		multiterm= 	mycas._multiterm
		outparent= 	mycas.outparent
		outterms= 	mycas.outterms
		outpos=		mycas.outpos
		outchild=	mycas.outchild;
	svd	
		k=			100
		outdocpro=	mycas.outdocpro keepvariables= (id)
		outtopics=	mycas.outtopics
		svdu=  		mycas.outsvdu;
run;

My primary indication that it is not working is that the term 'dont know' is not in mycas.outterms table. My secondary indication is that I can't get my topic assignments to change when I specify multi term phrases.

proc sql;
	select * from mycas.outterms where term = 'dont know';
quit;

Re: Specify multi word phrases in proc textmine

sbxkoenk — Mon, 20 May 2024 12:56:46 GMT

@supp wrote:

Hmmmm.
Can you try

dont know :3:Prep

... instead of the way you entered it above?

Koen

Re: Specify multi word phrases in proc textmine

supp — Mon, 20 May 2024 14:51:37 GMT

The formatting you suggested might be the solution. I put a space after the term, then no other spaces.

I now see the term 'dont know' in my outterms CAS table!

I will examine the results a bit more but this is progress!