Hi everyone,   my target variable is a binary variable and my task is the classic two-class classification problem. My colleague built both logistic regression model (using "Regression" node) and PLS logistic regression model (using "PLS" node) on our dataset. My question is:   Since PLS regression is a kind of many-to-many regression technique which combines the essence of principal component analysis, when there is only one respondent variable y (y ~ B(p), Bernoulli distribution) and PLS regression is applied, will we get the same or (at least) a very similar/comparable result to principal component regression?   Many thanks to all of you! ... View more

Hi everyone,   All discussions following are based on SAS Enterprise Miner. During the process of building different models I found that KNN is the most time-consuming method. For example, if the Neural Network takes few minutes to complete, KNN will take even several hours to finish modeling based on the same training and validation sample. I decide not to include KNN as a workable solution on our project, but in order to at least "get some results", I tend to apply it to smaller subsamples of both EXISTING training and validating datasets. To be specific,   (1) I have already done Partition, Metadata (role assignment), Replacement, Impute, PCA before KNN and I want to apply the results of PCA (i.e. the generated principal components) to KNN method; (2) Between PCA node and KNN node, I would like to resample in both training and validating datasets with the same proportion, e.g. 10% of the exported PCA training dataset and 10% of the exported PCA validating dataset; (3) SAS Code node will not be considered, since it is not easy to operate with to other users.   This is really a difficult problem for me. If any of you have any ideas, you are welcome to discuss with me and I will really appreciate it. Thanks very much! ... View more

Hi everyone,   At present I am trying different models on my dataset and the task is achieving two-class classification. I have made some findings so far: (1) Although SVM, Random Forest and Neural Network perform well with a pretty nice validating AUC (range from 0.87 to 0.9), the traditional Logistic Regression is also a good model with a validating AUC more than 0.89. (2) The results of the three machine learning methods are relative stable as long as I don't set some extreme values of hyper-parameters. (3) The distribution of the dependent variable in my dataset is extremely imbalanced with 98.5% observations in class 1 and 1.15% in class 2 and the second class is what I care about. The sample size is 100k (70k for training, 30k for validating) and the population is even larger.   My questions are: (1) Do you think that the imbalanced distribution of the dataset impairs the performance of model? (2) If the oversampling/ undersampling/ SMOTE and other resampling techniques are not allowed, how could I improve my model(from the perspective of AUC)? (3) How can I improve the prediction precision for the second class?   Thanks very much! ... View more

Hello everyone,   I just used the HP SVM node in SAS EM to build SVM model and it turned out to perform pretty well. But what is confusing to me is: In the output of SVM model , I see following information: Training Results Inner Product of Weights 72.2366916 Bias 5.27294747 Total Slack (Constraint Violations) 2962.97082 Norm of Longest Vector 12.6467077 Number of Support Vectors 69998 Number of Support Vectors on Margin 0 Maximum F 24.7895353 Minimum F 0.04925285 Number of Effects 18 Columns in Data Matrix 18 Columns in Kernel Matrix 190 (1) I don't know why it reports that "Number of Support Vectors" is 69998, which is just the size of my training dataset. That is obviously impossible that the model uses all observations in training dataset as support vectors since the AUC on training dataset is NOT equal to 1 and there is very slight overfitting situation.   (2) By the way, is there anyone who could tell me, that how can I see the REAL number of support vectors that the model uses?   Thanks very much. ... View more

Hello everyone,   I am trying to use SAS Code node with proc hpsplit to achieve hyperparameter-tuning of decision trees in SAS Enterprise Miner. This works and my codes so far are as following:   %macro DTStudy (maxbranch=2, maxdepth=5, minleafsize=20); %let branchTries = %sysfunc(countw(&maxbranch.)); %let depthTries = %sysfunc(countw(&maxdepth.)); %let leafsizeTries = %sysfunc(countw(&minleafsize.)); %do i = 1 %to &branchTries.; %do j = 1 %to &depthTries.; %do k = 1 %to &leafsizeTries.; %let thisBranch = %sysfunc(scan(&maxbranch.,&i)); %let thisDepth = %sysfunc(scan(&maxdepth.,&j)); %let thisLeafsize = %sysfunc(scan(&minleafsize.,&k)); proc hpsplit data=&em_import_data maxbranch=&thisBranch. maxdepth=&thisDepth. nsurrogates=4 minleafsize=&thisLeafsize. mincatsize=2 assignmissing = similar; input %EM_INTERVAL_INPUT /level=interval; target %EM_TARGET / level=nominal; criterion gini; /*prune misc / N <= 6;*/ partition fraction(validate=0.3 seed=12345); prune ase; score out=score_b&thisBranch._d&thisDepth._l&thisLeafsize.; run; /* Add the value of branchToTry/depthToTry/leafsizeToTry for these fit stats */ data score_b&thisBranch._d&thisDepth._l&thisLeafsize.; length branchToTry $ 8; length depthToTry $ 8; length leafsizeToTry $ 8; set score_b&thisBranch._d&thisDepth._l&thisLeafsize.; branchToTry = "&thisBranch."; depthToTry = "&thisDepth."; leafsizeToTry = "&thisLeafsize."; run; /* Append to the single cumulative fit statistics table */ proc append base=fitStats data=score_b&thisBranch._d&thisDepth._l&thisLeafsize.; run; %end; %end; %end; %mend DTStudy; %DTStudy(maxbranch=2 3 4 5, maxdepth=5 10 12 15, minleafsize=10 20 30 40); %em_register(type=Data,key=fitStats); data &em_user_fitStats; set fitStats; run; %em_report(viewType=data,key=fitStats,autodisplay=y); My question is: does there exist an option to limit the minimal sample size of a node to consider further splitting? For example, you can set the "Split size" in the options of Decision Tree node. According to my knowledge, it is a relative important hyperparameter when the sample size is large (which is my situation).   I would really appreciate it if anyone could give me some advice and you are also welcome to discuss this problem with me. Thank you very much. ... View more