About Rick_SAS

Rick_SAS · ‎02-27-2012

I have been summoned.... Interesting. I wasn't aware of this behavior before, but from some experimentation I discovered that you can get PGM1 to give the same answer as PGM2 if you invoke ODS GRAPHICS ON and ask for PLOTS(ONLY)=ROCPLOT. This tell us that creating a ROC plot is changing the AUC algorithm somehow, and that the ROC statement changes the algorithm in the same way. By scanning the LOGISTIC doc, I narrowed in on either the ROCEPS= or BINWIDTH= option as the option that is causing the change. Further experimentation reveals that it is the BINWIDTH= option that is causing the difference in the two PROC runs. You can read about the reasons in the doc or in this USAGE NOTE, but I'll summarize what I think is happening in a few sentences: In the "old days" PROC LOGISTIC used an estimate of the AUC that corresponds to BINWIDTH=0.002. When ODS graphics came around and the ROC statement was added, computational power had increased that it was feasible to compute an exact AUC, so BINWIDTH=0 is used for these new features. But because SAS doesn't like to make changes that might affect someone's benchmarks/existing programs, the default value of 0.002 was retained for the case when ODS graphics are off and you don't use the ROC statement. So the answer is: PGM1 is an estimate, PGM2 is exact, and you can get PGM1 to give the exact answer by putting BINWIDTH=0 on the MODEL stmt. Rick Message was edited by: Rick Wicklin

Rick_SAS · ‎02-17-2012

You should read Paul Allison's books and papers. Start with Imputation of categorical variables with PROC MI

Rick_SAS · ‎02-17-2012

To clarify: Do you have missing values in categorical and continuous regressors? Or are you saying that the ordinal response variable has missing values?

Rick_SAS · ‎02-16-2012

The distributions in GENMOD are here: http://support.sas.com/documentation/cdl/en/statug/63962/HTML/default/viewer.htm#statug_genmod_sect037.htm

Rick_SAS · ‎02-15-2012

You can download Chap 2 for free at http://blogs.sas.com/content/iml/2010/11/15/free-chapter-getting-started-with-sasiml/ See also the blog post "How to learn SAS/IML: Five resources for the beginner."

Rick_SAS · ‎02-09-2012

Hold on. I thought the poster wants a numerical solution for this nonlinear system of equations. Write the expression on the left as F_t(R_e). Then the problem is solving the system F_t(R_e) = P_t for t=1..T. Now presumably there is no value of R_e that solves this exactly for all measured values of the data, so you want to find the value of R_e that best fits the data according to some criterion (for example, least squares). It seems like you can solve this problem in SAS. Maybe PROC MODEL? If not, please explain what I'm missing,

Rick_SAS · ‎02-03-2012

I suspect they are just plugging into the formula avg +/- t(1-alpha) s/sqrt(N) and since s=0, the CI is assigned zero width. It's really not clear to me what the correct answer should be for these degenerate data. Both can be justified. Let's see how other statisticians weigh-in.

Rick_SAS · ‎02-03-2012

Impressive results! Congratulations, and thanks for sharing!

Rick_SAS · ‎02-03-2012

I think it comes out of the derivation of the CLM formula. You derive the formula by looking at the expression t = (sampleAverage - populationMean)/ (sampleStdDev/sqrt(N)) You then argue that if N is large and the x are normally distributed (yada, yada, yada) then the statistics has a certain distribution. When you have constant data, the sample std dev is 0, and therefore the expression is undefined.

Rick_SAS · ‎02-03-2012

A surprisingly subtle question. For constant data, I think the official answer is undefined (or missing). I'll try to find a reference when I get into the office. On the other hand, the LIMIT of this situation is zero, which is probably why you are writing. In other words, if your data are {0, delta}, then the width of the CLM approaches zero. You can see this graphically by running the following SAS code: data a; keep sample x; do y = 1 to 0 by -0.05; sample + 1; x = 0; output; x = y; output; end; sample + 1; x=0; output; x=0; output; /* exact zero */ run; proc means data=a noprint; by sample; var x; output out=out range=range lclm=lclm mean=mean uclm=uclm; run; proc sgplot data=out; band x=range upper=uclm lower=lclm; run;

Rick_SAS · ‎02-02-2012

One way is to transpose the data so that each category become a variable, and then use the SGPLOT DENSITY stmt. To transpose the data, see http://blogs.sas.com/content/iml/2011/11/04/reshape-data-so-that-each-category-becomes-a-new-variable/ Here's an example: data combo; keep Females Males; merge sashelp.class(where=(sex="F") rename=(height=Females)) sashelp.class(where=(sex="M") rename=(height=Males)); run; proc sgplot data=combo; density Females / legendlabel="Female"; density Males / legendlabel="Male"; xaxis label="Height"; run;

Rick_SAS · ‎02-02-2012

You can get a COMPARATIVE panel of histograms by using the CLASS stmt instead of the BY stmt: CLASS location; If this isn't sufficient, then I ask whether you just want the normal curves by themselves (in a single plot), or whether you want the densities fpor each group overlaid on a histogram. If on a histogram, then are you asking for the histogram of ALL the data, across all groups?

Rick_SAS · ‎02-02-2012

You didn't ask a question, so I can't tell whether you are saying "here is how to do this" or whether you are asking "is this the best way to do this." If the former, yes, this is a good way to compute cumulative sums across rows of a matrix.It is a clever algorithm. If you are asking whether this is the fastest way, I'll offer two observations: you can use the LAG function (introduced in SAS/IML 9.22) to speed up Step2, and you don't need the REPEAT function when forming the OUTPUT matrix. Thus the following code is slightly more efficient: start rs(x); Step1 = cusum(x); Step2 = lag(Step1[,ncol(Step1)], 1); /* returns missing in Step2[1] */ Step2[1] = 0; /* replace missing value with 0 */ Output = Step1 - Step2; return( Output); finish; Unless your matrices are huge, it doesn't really matter which module you run. In fact, unless the number of rows is huge, you can just use a simple DO loop and the computation is just as fast: Output = j(nrow(x), ncol(x)); do i = 1 to nrow(x); Output[i,] = cusum(x[i,]); end;

Rick_SAS · ‎02-01-2012

It might, if the time series aren't too wild and noisy. You need to do (1) because if there are variables that contain only a few nonmissing observations, those should be dropped from the analysis. Using PROC EXPAND (or any interpolation/imputation scheme) on data that has only a few valid points is not a good idea.

Rick_SAS · ‎02-01-2012

Use this example that I created: data Targets; input Target @@; /* NYC CHICAGO DALLAS */ datalines; 10027 60601 75211 ; data Cities; input City @@; datalines; 96106 49039 01341 13464 40206 15281 31515 67859 64465 80234 25672 45340 26266 77641 97306 92885 57580 36312 95070 29940 25165 16424 39423 76528 06611 ; proc sql; /* combine the two tables */ create table All as select * from Targets, Cities; quit; data dist; set All; dist = zipcitydistance(Target, City); run;

Online Status	Online
Date Last Visited	2 hours ago