About mcs

mcs · ‎12-14-2016

Thanks for the reply. I'm running SAS/IML 13.2 on Windows 7. OK, I did this. It shaved 20 seconds or so off the two-minute run time. The wbv parameter vector is the thing being optimized by proc nlpnra, and I think it has to be passed from nlpnra to ml as a vector. When I try to pass it as a matrix, I get ERROR: NLPNRA call: Error in argument X0. OK, I did this. Surprisingly, it increased the run time by more than a minute. My matrices aren't that big. My batch size (rows of x) and number of principal components (columns of x) will both be under 1000. Regarding 3., I must be doing something wrong. Here's my do loop. wbv = j(1,&pg*11,0.1); do i = 1 to &nb; x[,1:&pc] = train[rows[i,],2:&pc+1]; yp[,unique(train[rows[i,],1])+1] = design(train[rows[i,],1]); call nlpnra(rc,xrv,'ml',wbv); wbv = xrv; xr[i,] = xrv; end; Commenting out the line wbv = xrv, so that wbv always starts as a constant matrix of 0.1 instead of starting at the last solution, speeds up the code instead of slowing it down. Why would that be?

mcs · ‎12-13-2016

I'd like to implement TensorFlow's MNIST for ML Beginners in SAS. https://www.tensorflow.org/tutorials/mnist/beginners/ It classifies handwritten digits based on pixel intensities. Their model is y = softmax(Wx + b), where y = output vector of digit probabilities W = weight matrix x = pixel intensities b = bias vector They optimize W and b by minimizing cross-entropy, H = -Σy'log(y). I've got some working SAS code, but it's not fast enough to do what I want yet. Using 10 principal components (instead of all the pixel values), and 10 training batches of 100 observations each, my code runs in about 2 minutes. I'd like to increase each of those parameteres by a factor of 10. Here's the part of my code implementing their model.. wbv = j(1,10*&pc+10,0.1); /* 10 weights per pc + 10 bias, vector format */ start ml(wbv) global(x,yp); /* x is pc scores, yp is 1-hot digit labels */ wb = shape(wbv,&pc+1,10); /* weight and bias, matrix format */ logy = x*wb-log(exp(x*wb)[,+]); /* y is softmax of evidence */ ce = (-yp#logy)[:,+]; /* mean cross-entropy */ return ce; finish; I'm using call nlpnra(rc,xrv,'ml',wbv) as the optimization routine. Is there an easy way to get a big speedup?

mcs · ‎12-09-2016

I have an nx1 column vector, each element e_n an integer between 1 and p. I want to convert it to an nxp matrix with a 1 as the e_nth element of each row and the rest 0. have = {6,9,1,...} want = {0 0 0 0 0 1 0 0 0 ... , 0 0 0 0 0 0 0 0 1 ... , 1 0 0 0 0 0 0 0 0 ... , ... } What's the easiest way to do that?

mcs · ‎11-11-2016

I think I found the answer. In the selection method options, L1=0 requires stop=L1. When I changed stop=cv to stop=l1, the ridge regression gave a different model than the LASSO.

mcs · ‎11-11-2016

I'd like to use proc glmselect to compare ridge regresssion and LASSO on the same data. The documentation seems to say that selection=elasticnet with L1=0 is euivalent to ridge regression. proc glmselect data=train plots=all; class private; model apps = private accept--grad_rate / selection=elasticnet(choose=cv l1=0 stop=cv); score data=test p out=ridge; proc glmselect data=train plots=all; class private; model apps = private accept--grad_rate / selection=lasso(choose=cv stop=cv); score data=test p out=lasso; The data is the College.csv file from http://www-bcf.usc.edu/~gareth/ISL/data.html, split into training and test sets. The two models selected are exactly the same. (I also get the same model if I use elasticnet with L2=0.) Why is that? I expected ridge regression and LASSO to produce slightly different models since they use different constraints.

mcs · ‎09-21-2016

Thanks. I have 9.4 and was able to get the GROUPLP and GROUPLC options to work for me. I have a couple of questions about fine-tuning the graph. How can I control the colors of the lines? I have three colors in my graph, and the default seems to be blue, red, green. I would prefer red, blue, green in that order. Also, how can I control the entries in the legend? It's currently using the values of the GROUPLP and GROUPLC variables. I would like to specify more easily-readable strings.

mcs · ‎09-20-2016

> Did you try with the names of both graphs on the Keylegend statement? Yes. That just doubles the group names, i.e., the legend shows solid lines labeled by the values of the group=variable, and then dashed lines labeled by the same thing.

mcs · ‎09-20-2016

I'm trying to make a graph that compares actual to predicted lapse rates. proc sgplot; xaxis label='Age' values=(20 to 64 by 2); yaxis label='Lapse Rate' tickvalueformat=percent6. max=1; keylegend 'act'; vline age / response=lapse_rate group=dur_grp name='act' legendlabel='Actual Lapse Rate'; vline age / response=pred_rate group=dur_grp lineattrs=(pattern=dash) name='pred' legendlabel='Predicted Lapse Rate'; run; Since I'm using the group= option on the vline statements, the legendlabel= option has no effect. I'd like viewers to know that the solid lines are actual and the dashed lines are predicted. How can I get that information into a legend or a label somehow? Martin

mcs · ‎07-13-2016

I haven't used either of those before, and after a quick look at the documentation, I couldn't figure out how to make them do what I want. Can you explain how clustering lets me classify digits? I assume I would cluster the training dataset and then somehow use the output to score the test dataset, but I don't understand the details. Specifically, how would I use the known value of the digit in the training dataset?

mcs · ‎07-12-2016

I've just begun working my way through the exercises in The Elements of Statistical Learning. Exercise 2.8 asks you to use k-nearest neighbors to classify scanned zipcode digits from greyscale values (gs1-gs256). Here's some of my code. %macro knn; %do i = 1 %to 5; %let k = %scan(1 3 5 7 15,&i); proc discrim data=train method=npar k=&k out=train_k&k._out(keep=digit _into_) testdata=test testout=test_k&k._out(keep=digit _into_); class digit; var gs1-gs256; run; %end; %mend; %knn; Using just digits 2 and 3, I get error rates on the test datasets (available at the book website) between 6% (for k1) up to 10% (for k15). Those don't agree with a couple of solutions on the web. Andrew Tulloch shows error rates between 2% and 4%, while Weatherwax and Epstein have error rates between 9% and 11%. Is there anyone else who has done the exercise and can confirm which of the three answers (if any) is correct? Martin

mcs · ‎04-25-2016

I'm using PROC REG in SAS 9.4 with the default ODS Graphics. What's the easiest way to add a reference line to the fit plot?

mcs · ‎01-12-2016

I'd like to work with complex numbers in SAS/IML. In particular, I'd like to do option pricing in the Heston model as described in Kahl and Jaeckel. I assume the complex numbers will be represented as two-component matrices. I also assume I'll be using modules to do complex arithmetic, including multiplication and division, as well as things like arg(c), |c|, exp(c), ln(c), for complex c. Is there a standard implementation of these compex arithmetic modules that I can use, or will I have to write my own?

mcs · ‎10-27-2014

Thanks. That looks like it should work even when the bounds on s and k are large. I think a slight improvement would be to calculate the running variance as shown at http://www.johndcook.com/standard_deviation.html Initialize M1 = x1 and S1 = 0. For subsequent x's, use the recurrence formulas Mk = Mk-1+ (xk - Mk-1)/k Sk = Sk-1 + (xk - Mk-1)*(xk - Mk). For 2 ≤ k ≤ n, the kth estimate of the variance is s2 = Sk/(k - 1). Martin

mcs · ‎10-27-2014

Thanks for the reply. This works, but the range of k is, say, 1 to 100 instead of 1 to 2, it involves a lot of repeated code. Martin

mcs · ‎10-27-2014

data have; input sex $ age; cards; M 45 F 50 U 60 ; data mc; set have; array s(5,2); do i = 1 to 5; do j = 1 to 2; s(i,j) = age + i + j; end; end; array mx(2); array sx(2); do k = 1 to 2; mx(k) = mean(of s(*,k)); sx(k) = std(of s(*,k); end; run; Desired output: _N_ = 1 mx1 = mean(of (47,48,48,49,49)) = 48.2, similary for mx2 sx1 = std(of (47,48,48,49,49)) = 0.84, similary for sx2 _N_ = 2 mx1 = mean(of (52,53,53,54,54)) = 53.2, similary for mx2 sx1 = std(of (52,53,53,54,54)) = 0.84, similarly for sx2 _N_ = 3 etc. Instead I get the following syntax error. 21 mx(k) = mean(of (s(*,k)); - 22 76 ERROR 22-322: Syntax error, expecting one of the following: ), ], }. Martin

Online Status	Offline
Date Last Visited	‎05-11-2023 01:10 PM

Re: PROC GENMOD Error in computing inverse link function

Re: PROC GENMOD Error in computing inverse link function

Re: PROC GENMOD Error in computing inverse link function

Re: PROC GENMOD Error in computing inverse link function

PROC GENMOD Error in computing inverse link function

Re: %SYSFUNC and WHICHN

%SYSFUNC and WHICHN

Re: PROC SGMAP bubble plot

Re: PROC SGMAP bubble plot

PROC SGMAP bubble plot

Re: PROC SGMAP bubble plot

Re: PROC SGMAP bubble plot

Re: PROC SGMAP bubble plot

Re: PROC SGMAP bubble plot

Re: PROC SGMAP bubble plot

PROC GENMOD Error in computing inverse link function

Complex Numbers

TensorFlow MNIST

K-Nearest Neighbors for Zip Code Digits

Re: TensorFlow MNIST

TensorFlow MNIST

One-Hot Vectors

Re: Ridge Regression and LASSO

Ridge Regression and LASSO

Re: PROC SGPLOT legend labels with group option

Re: PROC SGPLOT legend labels with group option

PROC SGPLOT legend labels with group option

Re: K-Nearest Neighbors for Zip Code Digits

K-Nearest Neighbors for Zip Code Digits

PROC REG ODS Graphics Reference Line

Complex Numbers

Re: Mean and std of partial array

Re: Mean and std of partial array

Re: Mean and std of partial array

SAS Explore