I'm using text mining analyses of SAS Enterprise Miner 15.1. I wonder if we can drop the "text variable" in the "texttopic node", because it always spends a very long period of time for copying the "text variable" from the dataset. The dataset is around 30GB, the largest size variable belongs to the "text variable", because I'm parsing text data. If there is a way to drop this variable before creating the texttopic node, that will significantly reduce the required storage space and processing time. I think it's redundant to copy the "text variable" in the texttopic node, because the terms have been parsed by the "textparsing" and "textfilter" nodes? my standard procedure of performing text mining analysis: connect these nodes: sas dataset (with text variable and ID) & textparsing node (with startlist and other settings) & textfilter node (with synonym list and dictionary) & texttopic node (create around 200 single-term topics, no multi-term topic) Thank you so much!!
... View more