Hi,
I am running a bunch of code which is shared publicly by others and get the following error "(execution) Module not loaded, operation not available. operation : RAND at offset 10 column 16 operands : *LIT1142". I am totally new to sas and have no idea where the problem could come from. Any hints will be much appreciated!
210 *____________________________________________________________________________________________________ 211 212 This proc loops over all samples, calling the estimation routines. 213 _____________________________________________________________________________________________________; 214 options nonotes; 215 proc iml; 216 216 ! start main; 217 217 ! call streaminit(1234); 218 218 ! reset storage=this.imlstor; 218 ! * This contains the subroutines built in RollGibbsLibrary02.sas; 219 219 ! load; 220 221 221 ! reset printadv=1 log; 222 222 ! use temp.pys; 223 223 ! read all var {permno year kSample} where(nTradeDays>=60) into sample [colname=colSample]; 223 ! *my correction; 224 224 ! nSamples = nrow(sample); 225 225 ! print 'nSamples=' nSamples; 226 226 ! permno = sample[,1]; 227 227 ! year = sample[,2]; 228 228 ! kSample = sample[,3]; 229 229 ! outSet = j(1,7,.); 230 230 ! varnames ={'permno','year','kSample','c','beta','varu','sdu'}; 231 231 ! create this.gibbsOut from outSet [colname=varnames]; 9 The SAS System Sunday, August 8, 2021 02:08:00 AM 232 232 ! do iSample=1 to nSamples; 233 233 ! thisPermno = permno[iSample]; 234 234 ! thisYear = year[iSample]; 235 235 ! thisKSample = kSample[iSample]; 236 236 ! if mod(iSample,500)=1 then do; 237 237 ! t = time(); 238 238 ! ctime = putn(t,'time.'); 239 239 ! print ctime iSample '/' nSamples ': ' thisPermno thisYear thisKSample; 240 240 ! end; 241 241 ! if thisPermno>=&startPermno & thisYear>=1990 then do; 242 242 ! use temp.dsf where (permno=thisPermno & year=thisYear & kSample=thisKSample); 243 243 ! read all var {p pm q} into x [colname=colx]; 244 245 245 ! nSweeps = 1000; 246 246 ! regDraw = 1; 247 247 ! varuDraw = 1; 248 248 ! qDraw = 1; 249 249 ! nDrop = 200; 250 251 251 ! call RollGibbsBeta(parmOut, x[,1],x[,2],x[,3], nSweeps, regDraw, varuDraw, qDraw, 0,0,0,0); 252 253 253 ! p2 = parmOut[(nDrop+1):nSweeps,]; 254 254 ! p2 = p2 || sqrt(p2[,3]); 255 255 ! pm = p2[+,]/(nSweeps-nDrop); 256 256 ! outset = thisPermno || thisYear || thisKSample || pm; 257 *print outset; 258 258 ! setout this.gibbsOut; 259 259 ! append from outSet; 260 260 ! end; 261 261 ! end; 262 262 ! finish main; 263 run; 10 The SAS System Sunday, August 8, 2021 02:08:00 AM nSamples nSamples= 356031 ctime iSample nSamples thisPermno thisYear thisKSample 2:19:25 1 / 356031 : 10001 1986 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:21:07 501 / 356031 : 10034 1986 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:22:58 1001 / 356031 : 10078 1987 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:25:16 1501 / 356031 : 10121 1988 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:27:09 2001 / 356031 : 10153 1966 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:28:56 2501 / 356031 : 10196 1950 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:30:55 3001 / 356031 : 10233 1938 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:33:04 3501 / 356031 : 10268 1950 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:35:17 4001 / 356031 : 10308 2014 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:37:34 4501 / 356031 : 10362 2002 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:39:37 5001 / 356031 : 10401 1930 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:41:34 5501 / 356031 : 10443 2017 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:42:51 6001 / 356031 : 10487 1967 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:44:33 6501 / 356031 : 10527 1995 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:46:41 7001 / 356031 : 10571 2002 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:48:17 7501 / 356031 : 10620 1940 1 11 The SAS System Sunday, August 8, 2021 02:08:00 AM ctime iSample nSamples thisPermno thisYear thisKSample 2:50:41 8001 / 356031 : 10661 1998 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:52:46 8501 / 356031 : 10715 1988 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:53:56 9001 / 356031 : 10755 1994 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:56:07 9501 / 356031 : 10800 1996 1 ctime iSample nSamples thisPermno thisYear thisKSample 2:58:27 10001 / 356031 : 10853 2012 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:00:25 10501 / 356031 : 10890 1956 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:02:51 11001 / 356031 : 10933 1995 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:04:47 11501 / 356031 : 10984 1993 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:06:39 12001 / 356031 : 11033 1963 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:08:40 12501 / 356031 : 11078 1994 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:10:12 13001 / 356031 : 11135 1989 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:12:07 13501 / 356031 : 11174 2014 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:14:20 14001 / 356031 : 11244 1951 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:16:16 14501 / 356031 : 11293 1988 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:17:46 15001 / 356031 : 11335 1997 2 ctime iSample nSamples thisPermno thisYear thisKSample 3:19:40 15501 / 356031 : 11368 2018 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:22:04 16001 / 356031 : 11403 1994 1 12 The SAS System Sunday, August 8, 2021 02:08:00 AM ctime iSample nSamples thisPermno thisYear thisKSample 3:23:58 16501 / 356031 : 11447 1975 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:25:49 17001 / 356031 : 11490 1995 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:27:43 17501 / 356031 : 11535 1936 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:29:55 18001 / 356031 : 11599 1993 3 ctime iSample nSamples thisPermno thisYear thisKSample 3:32:18 18501 / 356031 : 11643 1991 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:34:23 19001 / 356031 : 11678 2001 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:36:34 19501 / 356031 : 11729 1996 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:38:31 20001 / 356031 : 11764 1988 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:41:03 20501 / 356031 : 11826 1930 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:43:01 21001 / 356031 : 11868 1995 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:45:34 21501 / 356031 : 11914 1951 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:48:02 22001 / 356031 : 11981 1926 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:50:21 22501 / 356031 : 12018 2012 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:52:11 23001 / 356031 : 12052 1994 2 ctime iSample nSamples thisPermno thisYear thisKSample 3:54:45 23501 / 356031 : 12079 1931 1 ctime iSample nSamples thisPermno thisYear thisKSample 3:56:25 24001 / 356031 : 12110 2002 1 ERROR: (execution) Module not loaded, operation not available. operation : RAND at offset 10 column 16 operands : *LIT1142 13 The SAS System Sunday, August 8, 2021 02:08:00 AM *LIT1142 1 row 1 col (character, size 7) uniform statement : ASSIGN at offset 10 column 1 traceback : module RANDSTDNORMT at offset 10 column 1 module MVNRNDT at offset 12 column 2 module ROLLGIBBSBETA at offset 39 column 4 module MAIN at line 251 column 4 264 quit;
Hello @tammytt ,
Welcome to the SAS communities!
In fact you are on the wrong board.
Well, you WERE on the wrong board as the thread is moved meanwhile by @Kurt_Bremser !
The programming board is more for Base SAS questions. Base SAS is a programming language with data steps (for data engineering) and procedure steps (for analyses) and global statements, including macro language.
SAS contains multiple / several languages, one of them is the Interactive Matrix Language (IML), the one you (try to) use.
The appropriate board for that matrix language (IML) is 'SAS/IML Software and Matrix Computations' under the Analytics heading. Your topic is on this board now, so all fine!
But OK, let's focus on your question now.
Are you trying to do 'Gibbs sampling'?
The MCMC (Markov chain Monte Carlo (MCMC)) procedure in SAS/STAT supports Gibbs sampling.
But you can also do it with PROC IML of course (although IML is mostly turned to if there is no readily available procedure for the algorithm you want to use).
I do not think the "RollGibbsLibrary02.sas" program that builds the subroutines is a SAS supplied program.
Apparently, there is a problem in module ROLLGIBBSBETA with the RAND function.
I guess in the function call, RAND('BETA', alpha, beta), there is a problem with alpha and / or beta. But we cannot see the RAND function in your code / log. So I even don't know if BETA is the actual distribution used!
Just FYI: it is recommended that you use RAND('BETA', alpha, beta), where (0.01 <= alpha <= 1.5e6) and (0.20 <= beta <= 1.5e6).
It is also possible that the module is just not loaded because it is not found or not compiled.
What SAS version are you using?
Please provide us with the log of this statement (that you submit):
%PUT &=sysvlong;
Any chance that you can get into contact with the author of that subroutine?
Kind regards,
Koen
Hello,
Adding upon my previous response (see above for that one).
This paper may be beneficial to you to understand what "your" IML code is doing.
Paper 3291 - 2015
Coding your own MCMC algorithm
Chelsea Loomis Lofland, University of California, Santa Cruz, CA
https://support.sas.com/resources/papers/proceedings15/3291-2015.pdf
If you are new to SAS, IML is not the easiest thing to start with. Because you should first master some basic concepts from Base SAS (like libraries and catalogs among others).
Do you happen to know R?
Knowing R may make it easier to master IML because SAS/IML and R operate rather similarly and treat vectors in the same manner with minor (well, "minor" is subjective of course) differences in syntax.
Good luck,
Koen
Hi Koen,
Thank you so much for your detailed guidance. Next time I will definitely put my question in the right board!
Unfortunately, I couldn't get touch with the author of the code.
I was using the SAS on the WRDS Cloud, as the code is based on the data on WRDS and this is also how the codes are designed. The SAS version there is:
SYSVLONG=9.04.01M7P080520
And I just checked the Rand function in module ROLLGIBBSBETA and seems like it is using the RAND('GAMMa', alpha) function (what I just searched). As you could see from the following code of ROLLGIBBSBETA, it is written as "rand('gamma',postAlpha)". I was wondering if it is just the problem of the lower case of 'gamma'. Please forgive me if it's a silly question.
*____________________________________________________________________________________________________ RollGibbsBeta: Estimate Roll model with Gibbs sampler The return argument is parmOut[nSweeps,3] Column 1: c Column 2: beta Column 3: varu ____________________________________________________________________________________________________; %let Infinity=1e30; %let eps=1e-30; start RollGibbsBeta(parmOut, p, pm, q, nSweeps, regDraw, varuDraw, qDraw, varuStart, cStart, betaStart, printLevel); nObs = nrow(p); if nrow(q)^=nObs | nrow(pm)^=nObs then do; print 'RollGibbsBeta length mismatch'; return; end; dp = p[2:nObs] - p[1:(nObs-1)]; if qDraw then do; * Initialize qs to price sign changes; qInitial = {1} // sign(dp); qInitial = qInitial # (q^=0); * Only initialize nonzero elements of q; q = qInitial; end; if varuStart<=0 then varuStart = 0.001; varu = varuStart; if cStart<=0 then cStart=0.01; c = cStart; if betaStart<=0 then betaStart=1; beta = betaStart; parmOut = j(nSweeps,3,.); do sweep=1 to nSweeps; dq = q[2:nObs] - q[1:(nObs-1)]; dpm = pm[2:nObs] - pm[1:(nObs-1)]; if regDraw then do; priorMu = {0,1}; postMu = priorMu; priorCov = diag({1,2}); postCov = priorCov; X = colvec(dq) || colvec(dpm); rc = BayesRegressionUpdate(priorMu, priorCov, dp, X, varu, postMu, postCov); if printLevel>=2 then print postMu postCov; coeffLower={0,-&Infinity}; coeffUpper=j(2,1,&Infinity); coeffDraw = mvnrndT(postMu, postCov, coeffLower, coeffUpper); if printLevel>=2 then print coeffDraw; c = coeffDraw[1]; beta = coeffDraw[2]; end; if varuDraw then do; u = dp - c*dq - beta*dpm; priorAlpha = 1.e-12; priorBeta = 1.e-12; postAlpha = .; postBeta = .; rc = BayesVarianceUpdate(priorAlpha, priorBeta, u, postAlpha, postBeta); x = (1/postBeta) * rand('gamma',postAlpha); varu = 1/x; sdu = sqrt(varu); if printLevel>=2 then print varu; end; if qDraw then do; qDrawPrintLevel = 0; p2 = p - beta*pm; call qDraw(p2, q, c, varu, qDrawPrintLevel); end; parmOut[sweep, 1]=c; parmOut[sweep, 2] = beta; parmOut[sweep, 3] = varu; end; finish RollGibbsBeta;
Thank you again for your kind help!
Best regards,
Tammy
Hello @tammytt ,
You are running (or WRDS Cloud is running) SAS 9.4 Maintenance Level 7 (build date: 05AUG2020).
That's the last SAS 9.4 before SAS VIYA, so that's good! I mean, SAS is at a good level.
If you are running on the WRDS Cloud (I presume that is : Wharton Research Data Services - University of Pennsylvania), does it mean that this
RollGibbsBeta
IML-module was written over there? If yes, @mkeintz can maybe help you out. I believe (not sure though) he's working over there.
Spelling GAMMA or gamma as GAMMa is perfectly fine. SAS (the programming language, not the data sets) is case insensitive in general (except for folder paths and filenames if you are in a UNIX environment and a few other exceptions).
Defining (and compiling) your IML-module each time again should be an effective strategy to have it always available.
So, I do not know what goes wrong in your case.
Maybe the numeric shape parameter 'postalpha' in the RAND('GAMMA', postalpha) is becoming too big? I do not know about an upper limit for this shape parameter but I can imagine problems occur at values > 1E20. Not sure though. I'm just guessing.
Can you run it (the concerned module) on sample data or with a limited amount of real data (or a limited amount of loops)? Have you run it successfully before?
For the particulars of the STORE / RESTORE ( / LOAD ) functionality and commands for storing modules (and matrices) in SAS/IML, I think @Rick_SAS is better placed to answer that.
Good luck,
Koen
Hi Koen,
Thanks for your help!
I haven't run it successfully before. But I do get the codes of testing the subroutines, which works just fine as following.
*______________________________________________________________________________________________ Test the routines ______________________________________________________________________________________________; proc iml; start main; call streaminit(1234); reset storage=this.imlstor; load; print 'Test of RandStdNorm:'; x =RandStdNormT(.5,1); print x; print 'Test of mvnrndT:'; mu={1 2}; cov={1 .5, .5 1}; vLower = {0 0}; vUpper = {&infinity &infinity}; x = mvnrndT(mu, cov, vLower, vUpper); print x; print 'Test of Bayesian normal regression update'; nObs = 1000; nv = 2; x = j(nObs,nv); u = j(nObs,1); sdu = 2; do t=1 to nObs; x[t,1] = 1; u = sdu*rand('normal'); do i=2 to nv; x[t,i] = rand('normal'); end; end; y = x * colvec((1+(1:nv))) + u; priorMu={0,0}; priorCov={&Infinity 0, 0 &Infinity}; dVar = sdu*sdu; postMu = 0; postCov = 0; rc = BayesRegressionUpdate(priorMu, priorCov, y, X, dVar, postMu, postCov); print postMu; print postCov; priorAlpha = 1.e-6; priorBeta = 1.e-6; postAlpha = 0; postBeta = 0; u = y - X*postMu; rc = BayesVarianceUpdate(priorAlpha, priorBeta, u, postAlpha, postBeta); print postAlpha; print postBeta; finish main; run; quit; proc iml; start main; call streaminit(1234); reset storage=this.imlstor; load; nObs = 250; sdm = sqrt(0.3##2 / 250); sdu = sqrt(0.2##2 / 250); print sdu; u = j(nObs,1,.); v = j(nObs,1,.); call randseed(12345); call randgen(u,'normal',0,sdu); call randgen(v,'normal',0,sdm); beta = 1.1; c = .04; q = j(nObs,1,.); call randgen(q,'uniform'); q = sign(q-0.5); probZero = 0.6; z = j(nObs,1,.); call randgen(z,'uniform'); q = (z>probZero)#q; p = j(nObs,1,.); m = 0; do t=1 to nObs; m = m + beta*v[t] + u[t]; p[t] = m + c*q[t]; end; pM = t(cusum(t(v))); nSweeps = 1000; regDraw = 1; varuDraw = 1; qDraw = 1; nDrop = 200; call RollGibbsBeta(parmOut, p, pm, q, nSweeps, regDraw, varuDraw, qDraw, 0,0,0,0); p2 = parmOut[(nDrop+1):nSweeps,]; p2 = p2 || sqrt(p2[,3]); pm = p2[+,]/(nSweeps-nDrop); print pm; finish main; run; quit;
Presumably, this program used to work? Can you tell us what has changed? Did you recently upgrade your version of SAS? Did you move to a new system (or change OS from Windows to Linux?) Are you running 9 or SAS Viya?
If you have recently upgraded SAS, then you need to re-store the modules (such as ROLLGIBBSBETA ) that are stored to this.IMLSTOR catalog. There should be a file somewhere that defines the modules and has a RESET STORAGE statement and a STORE statement. Find that file and rerun it on your new system, which will hopefully fix the problem. If not, then we may need to see the contents of that file.
Hi,
Thank you for your response!
I am running the code on the SAS of WRDS Cloud, as the code is based the data in WRDS. The SAS version there is
SYSVLONG=9.04.01M7P080520
I have tried to run it on the WRDS web SAS studio, but it just keeps running for like a full day and night without a final result, so I couldn't determine whether it used to work on the studio or not.
Each time I run the code, I would rerun the sas file which the ROLLGIBBSBETA module is in to get a new this.IMLSTOR file. And I assume this should works like "RESTORE", i am not sure though..
Can you post the code for the RANDSTDNORMT module? That seems to be the module that is reporting an error.
And another thing is because I have run the program for several times, in each new run I just created a new folder and copy my two .sas files to the this new directory (the one with the subroutine and the main one). Could that be a problem like chaos with old one?
I have pasted here the contents of the subroutine file, any hints will be greatly appreciated!
options nodate nocenter nonumber ps=70 ls=120 nomprint; libname this '.'; %let Infinity=1e30; %let eps=1e-30; *______________________________________________________________________________________________ Define subroutines that will be used in simulations. ______________________________________________________________________________________________; proc iml; start main; reset storage=this.imlstor; store module=_all_; remove module=main; show storage; finish main; *____________________________________________________________________________________________________ RollGibbsBeta: Estimate Roll model with Gibbs sampler The return argument is parmOut[nSweeps,3] Column 1: c Column 2: beta Column 3: varu ____________________________________________________________________________________________________; %let Infinity=1e30; %let eps=1e-30; start RollGibbsBeta(parmOut, p, pm, q, nSweeps, regDraw, varuDraw, qDraw, varuStart, cStart, betaStart, printLevel); nObs = nrow(p); if nrow(q)^=nObs | nrow(pm)^=nObs then do; print 'RollGibbsBeta length mismatch'; return; end; dp = p[2:nObs] - p[1:(nObs-1)]; if qDraw then do; * Initialize qs to price sign changes; qInitial = {1} // sign(dp); qInitial = qInitial # (q^=0); * Only initialize nonzero elements of q; q = qInitial; end; if varuStart<=0 then varuStart = 0.001; varu = varuStart; if cStart<=0 then cStart=0.01; c = cStart; if betaStart<=0 then betaStart=1; beta = betaStart; parmOut = j(nSweeps,3,.); do sweep=1 to nSweeps; dq = q[2:nObs] - q[1:(nObs-1)]; dpm = pm[2:nObs] - pm[1:(nObs-1)]; if regDraw then do; priorMu = {0,1}; postMu = priorMu; priorCov = diag({1,2}); postCov = priorCov; X = colvec(dq) || colvec(dpm); rc = BayesRegressionUpdate(priorMu, priorCov, dp, X, varu, postMu, postCov); if printLevel>=2 then print postMu postCov; coeffLower={0,-&Infinity}; coeffUpper=j(2,1,&Infinity); coeffDraw = mvnrndT(postMu, postCov, coeffLower, coeffUpper); if printLevel>=2 then print coeffDraw; c = coeffDraw[1]; beta = coeffDraw[2]; end; if varuDraw then do; u = dp - c*dq - beta*dpm; priorAlpha = 1.e-12; priorBeta = 1.e-12; postAlpha = .; postBeta = .; rc = BayesVarianceUpdate(priorAlpha, priorBeta, u, postAlpha, postBeta); x = (1/postBeta) * rand('gamma',postAlpha); varu = 1/x; sdu = sqrt(varu); if printLevel>=2 then print varu; end; if qDraw then do; qDrawPrintLevel = 0; p2 = p - beta*pm; call qDraw(p2, q, c, varu, qDrawPrintLevel); end; parmOut[sweep, 1]=c; parmOut[sweep, 2] = beta; parmOut[sweep, 3] = varu; end; finish RollGibbsBeta; *______________________________________________________________________________________________ call qDraw(p, q, c, varu, printLevel) makes new draws for q parameters: p column vector of trade prices q column vector of qs (REPLACED ON RETURN) c cost parameter varu variance of disturbance printLevel used to generate debugging output _______________________________________________________________________________________________; start qDraw(p, q, c, varu, printLevel); if nrow(p)^=nrow(q) then do; print "qDraw. p and q are of different lengths. p is " nrow(p) "x" ncol(p) "; q is " nrow(q) "x" ncol(q); abort; end; if nrow(c)^=1 then do; print "qDraw. c should be a scalar. It is " nrow(c) "x" ncol(c); abort; end; if nrow(varu)^=1 then do; print "qDraw. varu should be a scalar. It is " nrow(varu) "x" ncol(varu); abort; end; if printlevel>0 then print p q; qNonzero = q^=0; q = q || q; p2 = p || p; nSkip = 2; modp = colvec( mod(1:nrow(p),nSkip) ); ru = uniform(j(nrow(p),1,0)); do iStart=0 to (nSkip-1); if printLevel>0 then print iStart [l="" r="qDraw. iStart:" f=1.]; k = modp=iStart; jnz = loc( k & qNonzero ); * These are the q's we'll be drawing.; if printLevel>0 then print jnz [l="Drawing q's for t=" f=3.]; q[jnz,1] = 1; q[jnz,2] = -1; cq = c*q; v = p2 - cq; u = v[2:nrow(v),] - v[1:(nrow(v)-1),]; if printLevel>0 then print u [l="u:"]; s=(u##2)/(2*varu); if printLevel>0 then print s [l="s"]; sSum = (s//{0 0}) + ({0 0}//s); if printLevel>0 then print sSum [l="sSum (before reduction)"]; sSum = sSum[jnz,]; if printLevel>0 then print sSum [l="sSum (after reduction)"]; logOdds = sSum[,2] - sSum[,1]; * Make sure that we won't get overflow when we call exp(); logOkay = logOdds<500; Odds = exp(logOkay#logOdds); pBuy = Odds/(1+Odds); pBuy = logOkay#pBuy + ^logOkay; if printLevel>0 then print pBuy [f=e10.]; qknz = 1 - 2*(ru[jnz]>pBuy); q[jnz,1] = qknz; if istart<(nSkip-1) then q[jnz,2] = qknz; end; q = q[,1]; finish qDraw; *______________________________________________________________________________________________ rc = BayesVarianceUpdate(priorAlpha, priorBeta, u, postAlpha, postBeta) updates the variance posterior (inverted gamma) inputs: priorAlpha and priorBeta u vector of estimated disturbances postAlpha and postBeta are updated on return to the posterior values _______________________________________________________________________________________________; start BayesVarianceUpdate(priorAlpha, priorBeta, u, postAlpha, postBeta); postAlpha = priorAlpha + nrow(u)/2; postBeta = priorBeta + (u##2)[+]/2; return (0); finish BayesVarianceUpdate; *______________________________________________________________________________________________ rc = BayesRegressionUpdate(priorMu, priorCov, y, X, dVar, postMu, postCov) computes coefficient posteriors for normal Bayesian regression model inputs: priorMu and priorCov coefficient priors (mean and covariance) y column vector of l.h.s. values X matrix of r.h.s. variables dVar error variance (taken as given) postMu and postCov coefficient posteriors. rc is a return code (0 if okay) _______________________________________________________________________________________________; start BayesRegressionUpdate(priorMu, priorCov, y, X, dVar, postMu, postCov); if ncol(priorMu)^=1 then do; print "BayesRegressionUpdate. priorMu is " nrow(priorMu) "x" ncol(priorMu) " (should be a column vector)"; return(-1); end; if nrow(X)<ncol(X) then do; print "BayesRegressionUpdate. X is " nrow(X) "x" ncol(X); return (-1); end; if nrow(X)^=nrow(y) | ncol(y)^=1 then do; print "BayesRegressionUpdate. X is " nrow(X) "x" ncol(X) "; y is " nrow(y) "x" ncol(y); return (-1); end; if nrow(priorMu)^=ncol(X) then do; print "BayesRegressionUpdate. X is " nrow(X) "x" ncol(X) "; priorMu is " nrow(priorMu) "x" ncol(priorMu) " (not conformable)"; return (-1); end; if nrow(priorCov)^=ncol(priorCov) | nrow(priorCov)^=nrow(priorMu) then do; print "BayesRegressionUpdate. priorMu is " nrow(X) "x" ncol(X) "; priorCov is " nrow(priorCov) "x" ncol(priorCov); return (-1); end; covi = inv(priorCov); Di = (1/dVar)*(t(X)*X) + covi; D = inv(Di); dd = (1/dVar)*t(X)*y + covi*priorMu; postMu = D*dd; postCov = D; return (0); finish BayesRegressionUpdate; *______________________________________________________________________________________________ RandStdNormT(zlow,zhigh) returns a random draw from the standard normal distribution truncated to the range (zlow, zhigh). _______________________________________________________________________________________________; start RandStdNormT(zlow,zhigh); if zlow=-&Infinity & zhigh=&Infinity then return (normal(seed)); PROBNLIMIT = 6; if zlow>PROBNLIMIT & (zhigh=&Infinity | zhigh>PROBNLIMIT) then return (zlow+100*&eps); if zhigh<-PROBNLIMIT & (zlow=-&Infinity | zlow<-PROBNLIMIT) then return (zhigh-100*&eps); if zlow=-&Infinity then plow=0; else plow = probnorm(zlow); if zhigh=&Infinity then phigh=1; else phigh = probnorm(zhigh); p = plow + rand('uniform')*(phigh-plow); if p=1 then return (zlow + 100*eps); if p=0 then return (zhigh - 100*eps); return (probit(p)); finish RandStdNormT; *______________________________________________________________________________________________ mvnrndT(mu, cov, vLower, vUpper) returns a random draw (a vector) from a multivariate normal distribution with mean mu and covariance matrix cov, truncated to the range (vLower, vUpper). vLower and vUpper are vectors (conformable with mu) that specify the upper and lower truncation points for each component. _______________________________________________________________________________________________; start mvnrndT(mu, cov, vLower, vUpper); f = t(root(cov)); n = nrow(mu)*ncol(mu); eta = j(n,1,0); low = (vLower[1]-mu[1])/f[1,1]; high = (vUpper[1]-mu[1])/f[1,1]; eta[1] = RandStdNormT(low,high); do k=2 to n; etasum = f[k,1:(k-1)]*eta[1:(k-1)]; low = (vLower[k]-mu[k]-etasum)/f[k,k]; high = (vUpper[k]-mu[k]-etasum)/f[k,k]; eta[k] = RandStdNormT(low,high); end; return (colvec(mu)+f*eta); finish mvnrndT; run; quit;
Thanks for posting the module definitions.
This might be impossible to solve without the data and a complete log. I spent all morning looking at this, but I am unable to reproduce it on simulated data. You might need to contact SAS Technical Support and arrange to provide the data and the full log.
I have a theory, which I cannot test. I will assume that this program used to work, even though you have never gotten it to work. If this program runs on different data every day/week/month, it might be that the data has properties that the program can't handle. For example, the program contains the following logic:
242 use temp.dsf where (permno=thisPermno & year=thisYear & kSample=thisKSample); 243 read all var {p pm q} into x [colname=colx];
Because of the WHERE clause, the number of rows for X depends on the data. It is possible that X has only 1 or 2 rows, and that is causing the program to fail. For example, you posted a Test Program that works, but if you change the program so that the data has less than five rows, the program fails:
proc iml;
start main;
call streaminit(1234);
reset storage=this.imlstor;
load module=_all_;
/* the program WORKS if NOBS>=5, but fails for small NOBS */
nObs = 4; /* <== change this line */
sdm = sqrt(0.3##2 / 250);
sdu = sqrt(0.2##2 / 250);
print sdu;
u = j(nObs,1,.);
v = j(nObs,1,.);
call randseed(12345);
call randgen(u,'normal',0,sdu);
call randgen(v,'normal',0,sdm);
beta = 1.1;
c = .04;
q = j(nObs,1,.);
call randgen(q,'uniform');
q = sign(q-0.5);
probZero = 0.6;
z = j(nObs,1,.);
call randgen(z,'uniform');
q = (z>probZero)#q;
p = j(nObs,1,.);
m = 0;
do t=1 to nObs;
m = m + beta*v[t] + u[t];
p[t] = m + c*q[t];
end;
pM = t(cusum(t(v)));
nSweeps = 1000;
regDraw = 1;
varuDraw = 1;
qDraw = 1;
nDrop = 200;
call RollGibbsBeta(parmOut, p, pm, q, nSweeps, regDraw, varuDraw, qDraw, 0,0,0,0);
p2 = parmOut[(nDrop+1):nSweeps,];
p2 = p2 || sqrt(p2[,3]);
pm = p2[+,]/(nSweeps-nDrop);
print pm;
finish main;
run MAIN;
quit;
The error is different from the one that you report, but the fact is that the program's behavior might depend on the data. So my question is: Can you run this program on "old" data? Does it only fail on "current" data?
If you can find one data set for which it works and another for which it fails, this Community or SAS Technical Support might be able to help you discover what properties of the data are causing it to fail.
Thanks so much for your help!
Another information is although the module RollGibbsBeta didn't run till the end successfully, it did produce (a part of) "gibbsOut" file with 13962 rows and 7 columns, and the results seems quite appropriate with the output in c, beta, varu and sdu (at least looks like so). Could that mean the module RollGibbsBeta works fine at the beginning and some inappropriate data in the middle make it fail?
the gibbsOut output:
the main program:
*____________________________________________________________________________________________________ This proc loops over all samples, calling the estimation routines. _____________________________________________________________________________________________________; options nonotes; proc iml; start main; call streaminit(1234); reset storage=this.imlstor; * This contains the subroutines built in RollGibbsLibrary02.sas; load; reset printadv=1 log; use temp.pys; read all var {permno year kSample} where(nTradeDays>=60) into sample [colname=colSample]; *my correction; nSamples = nrow(sample); print 'nSamples=' nSamples; permno = sample[,1]; year = sample[,2]; kSample = sample[,3]; outSet = j(1,7,.); varnames ={'permno','year','kSample','c','beta','varu','sdu'}; create this.gibbsOut from outSet [colname=varnames]; do iSample=1 to nSamples; thisPermno = permno[iSample]; thisYear = year[iSample]; thisKSample = kSample[iSample]; if mod(iSample,500)=1 then do; t = time(); ctime = putn(t,'time.'); print ctime iSample '/' nSamples ': ' thisPermno thisYear thisKSample; end; if thisPermno>=&startPermno & thisYear>=1990 then do; use temp.dsf where (permno=thisPermno & year=thisYear & kSample=thisKSample); read all var {p pm q} into x [colname=colx]; nSweeps = 1000; regDraw = 1; varuDraw = 1; qDraw = 1; nDrop = 200; call RollGibbsBeta(parmOut, x[,1],x[,2],x[,3], nSweeps, regDraw, varuDraw, qDraw, 0,0,0,0); p2 = parmOut[(nDrop+1):nSweeps,]; p2 = p2 || sqrt(p2[,3]); pm = p2[+,]/(nSweeps-nDrop); outset = thisPermno || thisYear || thisKSample || pm; *print outset; setout this.gibbsOut; append from outSet; end; end; finish main; run; quit; options notes; proc print data=this.gibbsOut (obs=50); run; data this.crspGibbs2009; set this.gibbsOut; run;
Yes. That is possible. This is good info to know.
Because this is an MCMC computation, it is also possible that the issue is caused by a random number that is extreme and is causing the program to misbehave. It would be interesting to change
call streaminit(1234);
to
call streaminit(4321);
and see if the error occurs at the same location (that is, the same number of rows processed until it fails). If the error occurs at the same place, it is likely the data that is causing the problem. If it occurs at a different place (or doesn't occur at all), it might be an extreme value for a random number.
Registration is now open for SAS Innovate 2025 , our biggest and most exciting global event of the year! Join us in Orlando, FL, May 6-9.
Sign up by Dec. 31 to get the 2024 rate of just $495.
Register now!
Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.
Find more tutorials on the SAS Users YouTube channel.