Does it have to be SAS/IML?   In SAS/Stat you can code: Q-technique factor analysis (as opposed to R-technique factor analysis, the most common form of factor analysis)   See: Applied Clustering Techniques Course Notes http://www.sas.com/store/books/categories/course-notes/applied-clustering-techniques-course-notes/prodCN_E70486_en.html https://support.sas.com/edu/schedules.html?id=2756&ctry=us   Appendix B Fuzzy Clustering .................................................................................. B-1 B.1 Fuzzy Clustering ............................................................................................................. B-3 Demonstration: Q-Methodology ............................................................................ B-11     Here you will learn to find fuzzy clusters using Q-technique factor analysis, in which observations are assessed on their relative membership in a set of derived factors. The derived factors are the fuzzy clusters. Squaring the factor loadings reflects the degree of membership exhibited by each observation in the associated cluster. Q-technique analysis is particularly applicable to clinical research (bioinformatics), in which there are typically many variables and relatively few cases.   Koen     ... View more

Hello,   There will always be a trade-off of course between: - sensitivity = true positive rate (TPR) = hit rate = recall on the one hand AND - precision = positive predictive value (PPV) on the other hand   Changing the cut-off (decision threshold) in a particular direction will improve one of these 2, but will worsen the other. Inevitably. Also: The higher the false positive rate, the lower the precision.   Do you already use the Gains table and the Trade-off Plots in the Scorecard node?   The trade-off plots display the approval rate and bad rate against cutoff scores. In credit scoring, trade-off plots are used to show how the approval rate and the bad rate among the accepted applicants depend on the cutoff score. A good scorecard enables the choice of a cutoff score that corresponds to a relatively high approval rate with a relatively low bad rate.   The gains table shows you "Average Marginal Profit" and "Average Total Profit" per score bucket using "Revenue Accepted Good" and "Cost Accepted Bad" (specified by you in the properties). I think the online doc (accessible from within EMiner) provides you with all the formulas.   If you don't want to rely on the Scorecard node for choosing your cut-off you can always consider to use the cut-off node. It will choose the "best" cut-off probability according to the criterion of your choice (you can easily derive which score is mapped to it). For example: Kolmogorov–Smirnov    And important! Enterprise Miner allows decision processing. See SAS® Enterprise Miner™ 14.1 Extension Nodes: Developer's Guide. https://support.sas.com/documentation/cdl/en/emxndg/67980/PDF/default/emxndg.pdf Appendix 3 Predictive Modeling Decision Thresholds and Profit Charts (p. 178) The final classification of a new applicant in the class of good or the class of bad risks will be based on profit considerations.   Some people choose to optimize the F1-score as the best balance between sensitivity and precision. F1-score is the harmonic mean of precision and sensitivity See https://en.wikipedia.org/wiki/Precision_and_recall in case you want to maximize the F1-score, you can write an optimization to find the best cut-off or simply a simulation (let the cut-off vary between a start and a stop value by an increment and calculate the quality metrics that go with each particular cut-off). Then make a choice.   Good luck, Koen   ... View more

Hello,   If you are doubting between 2 k-values, you can use Beale's F-type statistic to determine the final number of clusters. It will tell you whether the larger solution is significantly better or not (in the latter case the solution with fewer clusters is preferable). This technique is discussed in the "Applied Clustering Techniques" course notes.   You can also try something relatively new. Tip: K-means clustering in SAS - comparing PROC FASTCLUS and PROC HPCLUS https://communities.sas.com/t5/SAS-Communities-Library/Tip-K-means-clustering-in-SAS-comparing-PROC-FASTCLUS-and-PROC/ta-p/221369   For numeric variables, PROC HPCLUS provides the convenient NOC=ABC option to auto-select the number of clusters k based on the aligned box criterion (ABC). For each k value from MINCLUSTERS (default to 2) to MAXCLUSTERS, ABC compares the within-cluster dispersion of the results to that of a simulated reference distribution, and selects a value of k where the within-cluster dispersions of the data results and the reference distribution differ greatly.   See also: Paper SAS313-2014 An Overview of Machine Learning with SAS® Enterprise Miner™ Patrick Hall, Jared Dean, Ilknur Kaynar Kabul, Jorge Silva SAS Institute Inc. https://support.sas.com/resources/papers/proceedings14/SAS313-2014.pdf Find HPCLUS and ABC (keywords).   PROC HPCLUS is one of many High-Performance Procedures in SAS Enterprise MIner 13.2 and beyond.   Cheers, Koen   ... View more

Like this.   The MAX function ignores null values and SAS missing values. So the missing is completely ignored. Thus: max does not return 5 because 5 > . , the missing is just not considered.   data ab; input a b; datalines; 1 . . 2 3 . . 4 5 . ; run; PROC SQL noprint; create table abc as select * , max(a,b) as c from ab; QUIT; /* end of program */     Koen ... View more

Hello,   This is producing what you want. But I completely agree with LinusH, ... you should create a report instead of 'enriching' your dataset with an extra line below each subject containing his/her/its average.   So I don't recommend using the last data step. You don't want to spoil a dataset by adding extra rows containing an average while the other rows contain raw scores.   data work.ab; infile datalines truncover; input Subj $ score; datalines; a 1 a a 3 a 4 b b 4 b b 5 ; run; proc means data=work.ab nway noprint; class Subj; var score; output out=work.absummary mean= n= nmiss= / autoname; run; PROC SQL noprint; create table work.abmerge as select a.* , b.* from work.ab a , work.absummary b where a.Subj = b.Subj order by a.Subj , a.Score; quit; data work.abmerge; length Subj $ 25; set work.abmerge; by Subj; if last.Subj then do; output; if score_N >=3 then score=Score_Mean; else score=.; Subj="Avg. of above Subject"; output; end; if index(Subj,'Avg')=0 then output; run; /* end of program */     Cheers, Koen ... View more

Hello,   Read this tip. Tip: How to interpret your SAS® Rapid Predictive Modeler results? and more specifically: Selected Variable Importance and Cross Tabulations Scorecard - Selected Variable Importance Interpretation - How does Rapid Predictive Modeler create the Cross Tabulations Scorecard? - Cross Tabulation Scorecard Interpretation Don't forget to read the comments as well.   https://communities.sas.com/t5/SAS-Communities-Library/Tip-How-to-interpret-your-SAS-Rapid-Predictive-Modeler-results/ta-p/221282   Koen ... View more

Try these papers:   Paper 259-2009 Applications of the GLMSELECT Procedure for Megamodel Selection Robert A. Cohen, SAS Institute Inc., Cary, NC   https://support.sas.com/resources/papers/proceedings09/259-2009.pdf   Paper 207-31 (2006) Introducing the GLMSELECT PROCEDURE for Model Selection Robert A. Cohen, SAS Institute Inc. Cary, NC http://www2.sas.com/proceedings/sugi31/207-31.pdf   and a recent (2014) paper/presentation as well. Model Selection for Linear Models with SAS/STAT Software Funda Gunes SAS Institute Inc. http://www.amstat.org/meetings/csp/2014/onlineprogram/handouts/T3-Handouts.pdf   Koen   ... View more

Hello,   I don't know much about GEE models, but you should know there is a third procedure that you can use for this (PROC GEE, the GEE Procedure). The GEE procedure was introduced in SAS/STAT 13.2.   Take care if you have a SAS/STAT release prior to 13.2 because then your GEE procedure is experimental! SAS/STAT(R) 14.1 User's Guide The GEE Procedure http://support.sas.com/documentation/cdl/en/statug/68162/HTML/default/viewer.htm#statug_gee_toc.htm   See also this paper: Paper SAS166-2015 Weighted Methods for Analyzing Missing Data with the GEE Procedure Guixian Lin and Robert N. Rodriguez, SAS Institute Inc. https://support.sas.com/resources/papers/proceedings14/SAS166-2014.pdf   Cheers, Koen     ... View more