topic Re: ERROR: (execution) Module not loaded, operation not available in SAS/IML Software and Matrix Computations

ERROR: (execution) Module not loaded, operation not available

tammytt — Sun, 08 Aug 2021 09:20:47 GMT

Hi,

I am running a bunch of code which is shared publicly by others and get the following error "(execution) Module not loaded, operation not available. operation : RAND at offset 10 column 16 operands : *LIT1142". I am totally new to sas and have no idea where the problem could come from. Any hints will be much appreciated!

210        *____________________________________________________________________________________________________
211        
212        	This proc loops over all samples, calling the estimation routines.
213        _____________________________________________________________________________________________________;
214        options nonotes;
215        proc iml;
216        	
216      !  start main;
217        	
217      !  call streaminit(1234);
218        	
218      !  reset storage=this.imlstor;
218      !                             	*	This contains the subroutines built in RollGibbsLibrary02.sas;
219        	
219      !  load;
220        
221        	
221      !  reset printadv=1 log;
222        	
222      !  use temp.pys;
223        	
223      !  read all var {permno year kSample} where(nTradeDays>=60) into sample [colname=colSample];
223      !                                                                                             *my correction;
224        	
224      !  nSamples = nrow(sample);
225        	
225      !  print 'nSamples=' nSamples;
226        	
226      !  permno = sample[,1];
227        	
227      !  year = sample[,2];
228        	
228      !  kSample = sample[,3];
229        	
229      !  outSet = j(1,7,.);
230        	
230      !  varnames ={'permno','year','kSample','c','beta','varu','sdu'};
231        	
231      !  create this.gibbsOut from outSet [colname=varnames];
9 The SAS System                                                                               Sunday, August  8, 2021 02:08:00 AM

232        	
232      !  do iSample=1 to nSamples;
233        		
233      !   thisPermno = permno[iSample];
234        		
234      !   thisYear = year[iSample];
235        		
235      !   thisKSample = kSample[iSample];
236        		
236      !   if mod(iSample,500)=1 then do;
237        			
237      !    t = time();
238        			
238      !    ctime = putn(t,'time.');
239        			
239      !    print ctime iSample '/' nSamples ': ' thisPermno thisYear thisKSample;
240        		
240      !   end;
241        		
241      !   if thisPermno>=&startPermno & thisYear>=1990 then do;
242        			
242      !    use temp.dsf where (permno=thisPermno & year=thisYear & kSample=thisKSample);
243        			
243      !    read all var {p pm q} into x [colname=colx];
244        			
245        			
245      !    nSweeps = 1000;
246        			
246      !    regDraw = 1;
247        			
247      !    varuDraw = 1;
248        			
248      !    qDraw = 1;
249        			
249      !    nDrop = 200;
250        		
251        			
251      !    call RollGibbsBeta(parmOut, x[,1],x[,2],x[,3], nSweeps, regDraw, varuDraw, qDraw, 0,0,0,0);
252        			
253        			
253      !    p2 = parmOut[(nDrop+1):nSweeps,];
254        			
254      !    p2 = p2 || sqrt(p2[,3]);
255        			
255      !    pm = p2[+,]/(nSweeps-nDrop);
256        			
256      !    outset = thisPermno || thisYear || thisKSample || pm;
257        			*print outset;
258        			
258      !    setout this.gibbsOut;
259        			
259      !    append from outSet;
260        		
260      !   end;
261        	
261      !  end;
262        	
262      !  finish main;
263        run;
10 The SAS System                                                                              Sunday, August  8, 2021 02:08:00 AM

           nSamples

nSamples=    356031

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:19:25         1 /    356031 :       10001      1986           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:21:07       501 /    356031 :       10034      1986           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:22:58      1001 /    356031 :       10078      1987           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:25:16      1501 /    356031 :       10121      1988           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:27:09      2001 /    356031 :       10153      1966           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:28:56      2501 /    356031 :       10196      1950           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:30:55      3001 /    356031 :       10233      1938           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:33:04      3501 /    356031 :       10268      1950           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:35:17      4001 /    356031 :       10308      2014           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:37:34      4501 /    356031 :       10362      2002           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:39:37      5001 /    356031 :       10401      1930           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:41:34      5501 /    356031 :       10443      2017           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:42:51      6001 /    356031 :       10487      1967           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:44:33      6501 /    356031 :       10527      1995           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:46:41      7001 /    356031 :       10571      2002           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:48:17      7501 /    356031 :       10620      1940           1
11 The SAS System                                                                              Sunday, August  8, 2021 02:08:00 AM

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:50:41      8001 /    356031 :       10661      1998           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:52:46      8501 /    356031 :       10715      1988           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:53:56      9001 /    356031 :       10755      1994           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:56:07      9501 /    356031 :       10800      1996           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 2:58:27     10001 /    356031 :       10853      2012           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:00:25     10501 /    356031 :       10890      1956           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:02:51     11001 /    356031 :       10933      1995           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:04:47     11501 /    356031 :       10984      1993           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:06:39     12001 /    356031 :       11033      1963           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:08:40     12501 /    356031 :       11078      1994           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:10:12     13001 /    356031 :       11135      1989           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:12:07     13501 /    356031 :       11174      2014           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:14:20     14001 /    356031 :       11244      1951           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:16:16     14501 /    356031 :       11293      1988           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:17:46     15001 /    356031 :       11335      1997           2

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:19:40     15501 /    356031 :       11368      2018           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:22:04     16001 /    356031 :       11403      1994           1
12 The SAS System                                                                              Sunday, August  8, 2021 02:08:00 AM

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:23:58     16501 /    356031 :       11447      1975           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:25:49     17001 /    356031 :       11490      1995           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:27:43     17501 /    356031 :       11535      1936           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:29:55     18001 /    356031 :       11599      1993           3

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:32:18     18501 /    356031 :       11643      1991           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:34:23     19001 /    356031 :       11678      2001           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:36:34     19501 /    356031 :       11729      1996           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:38:31     20001 /    356031 :       11764      1988           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:41:03     20501 /    356031 :       11826      1930           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:43:01     21001 /    356031 :       11868      1995           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:45:34     21501 /    356031 :       11914      1951           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:48:02     22001 /    356031 :       11981      1926           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:50:21     22501 /    356031 :       12018      2012           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:52:11     23001 /    356031 :       12052      1994           2

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:54:45     23501 /    356031 :       12079      1931           1

ctime      iSample    nSamples    thisPermno  thisYear thisKSample

 3:56:25     24001 /    356031 :       12110      2002           1
ERROR: (execution) Module not loaded, operation not available.

 operation : RAND at offset  10 column  16
 operands  : *LIT1142

13 The SAS System                                                                              Sunday, August  8, 2021 02:08:00 AM

*LIT1142      1 row       1 col     (character, size 7)

 uniform

 statement : ASSIGN at offset  10 column   1
 traceback : module RANDSTDNORMT at offset  10 column   1
             module MVNRNDT at offset  12 column   2
             module ROLLGIBBSBETA at offset  39 column   4
             module MAIN at line 251 column 4

264        quit;

Re: ERROR: (execution) Module not loaded, operation not available

Ksharp — Sun, 08 Aug 2021 11:37:59 GMT

From your LOG, it is saying module like "call RollGibbsBeta()" NOT loading, so firstly check its location .
And it is IML code ,better post it at IML forum :
https://communities.sas.com/t5/SAS-IML-Software-and-Matrix/bd-p/sas_iml

and calling @Rick_SAS

Re: ERROR: (execution) Module not loaded, operation not available

Kurt_Bremser — Sun, 08 Aug 2021 11:45:08 GMT

Moved the thread.

Re: ERROR: (execution) Module not loaded, operation not available

sbxkoenk — Sun, 08 Aug 2021 12:16:15 GMT

Hello @tammytt ,

Welcome to the SAS communities!

In fact you are on the wrong board.
Well, you WERE on the wrong board as the thread is moved meanwhile by @Kurt_Bremser !

The programming board is more for Base SAS questions. Base SAS is a programming language with data steps (for data engineering) and procedure steps (for analyses) and global statements, including macro language.

SAS contains multiple / several languages, one of them is the Interactive Matrix Language (IML), the one you (try to) use.
The appropriate board for that matrix language (IML) is 'SAS/IML Software and Matrix Computations' under the Analytics heading. Your topic is on this board now, so all fine!

But OK, let's focus on your question now.

Are you trying to do 'Gibbs sampling'?
The MCMC (Markov chain Monte Carlo (MCMC)) procedure in SAS/STAT supports Gibbs sampling.

But you can also do it with PROC IML of course (although IML is mostly turned to if there is no readily available procedure for the algorithm you want to use).

I do not think the "RollGibbsLibrary02.sas" program that builds the subroutines is a SAS supplied program.
Apparently, there is a problem in module ROLLGIBBSBETA with the RAND function.
I guess in the function call, RAND('BETA', alpha, beta), there is a problem with alpha and / or beta. But we cannot see the RAND function in your code / log. So I even don't know if BETA is the actual distribution used!
Just FYI: it is recommended that you use RAND('BETA', alpha, beta), where (0.01 <= alpha <= 1.5e6) and (0.20 <= beta <= 1.5e6).

It is also possible that the module is just not loaded because it is not found or not compiled.

What SAS version are you using?

Please provide us with the log of this statement (that you submit):

%PUT &=sysvlong;

Any chance that you can get into contact with the author of that subroutine?

Kind regards,
Koen

Re: ERROR: (execution) Module not loaded, operation not available

sbxkoenk — Sun, 08 Aug 2021 12:30:02 GMT

Hello,

Adding upon my previous response (see above for that one).

This paper may be beneficial to you to understand what "your" IML code is doing.

Paper 3291 - 2015
Coding your own MCMC algorithm
Chelsea Loomis Lofland, University of California, Santa Cruz, CA

https://support.sas.com/resources/papers/proceedings15/3291-2015.pdf

If you are new to SAS, IML is not the easiest thing to start with. Because you should first master some basic concepts from Base SAS (like libraries and catalogs among others).

Do you happen to know R?

Knowing R may make it easier to master IML because SAS/IML and R operate rather similarly and treat vectors in the same manner with minor (well, "minor" is subjective of course) differences in syntax.

Good luck,

Koen

Re: ERROR: (execution) Module not loaded, operation not available

Rick_SAS — Sun, 08 Aug 2021 13:31:58 GMT

Presumably, this program used to work? Can you tell us what has changed? Did you recently upgrade your version of SAS? Did you move to a new system (or change OS from Windows to Linux?) Are you running 9 or SAS Viya?

If you have recently upgraded SAS, then you need to re-store the modules (such as ROLLGIBBSBETA ) that are stored to this.IMLSTOR catalog. There should be a file somewhere that defines the modules and has a RESET STORAGE statement and a STORE statement. Find that file and rerun it on your new system, which will hopefully fix the problem. If not, then we may need to see the contents of that file.

Re: ERROR: (execution) Module not loaded, operation not available

tammytt — Sun, 08 Aug 2021 16:02:40 GMT

Hi Koen,

Thank you so much for your detailed guidance. Next time I will definitely put my question in the right board!

Unfortunately, I couldn't get touch with the author of the code.

I was using the SAS on the WRDS Cloud, as the code is based on the data on WRDS and this is also how the codes are designed. The SAS version there is:

SYSVLONG=9.04.01M7P080520

And I just checked the Rand function in module ROLLGIBBSBETA and seems like it is using the RAND('GAMMa', alpha) function (what I just searched). As you could see from the following code of ROLLGIBBSBETA, it is written as "rand('gamma',postAlpha)". I was wondering if it is just the problem of the lower case of 'gamma'. Please forgive me if it's a silly question.

*____________________________________________________________________________________________________

	RollGibbsBeta: Estimate Roll model with Gibbs sampler
	
	The return argument is parmOut[nSweeps,3]
	Column 1:	c
	Column 2: beta
	Column 3: varu
____________________________________________________________________________________________________;


%let Infinity=1e30;
%let eps=1e-30;

start RollGibbsBeta(parmOut, p, pm, q, nSweeps, regDraw, varuDraw, qDraw, varuStart, cStart, betaStart, printLevel);
	nObs = nrow(p);
	if nrow(q)^=nObs | nrow(pm)^=nObs then do;
		print 'RollGibbsBeta length mismatch';
		return;
	end;
	dp = p[2:nObs] - p[1:(nObs-1)];
	
	if qDraw then do; 	*	Initialize qs to price sign changes;
		qInitial = {1} // sign(dp);
		qInitial = qInitial # (q^=0);	*	Only initialize nonzero elements of q;
		q = qInitial;
	end;

	if varuStart<=0 then varuStart = 0.001;
	varu = varuStart;
	if cStart<=0 then cStart=0.01;
	c = cStart;
	if betaStart<=0 then betaStart=1;
	beta = betaStart;
	
	parmOut = j(nSweeps,3,.);

	do sweep=1 to nSweeps;

		dq =  q[2:nObs] - q[1:(nObs-1)];
		dpm = pm[2:nObs] - pm[1:(nObs-1)];

		if regDraw then do;
			priorMu = {0,1};
			postMu = priorMu;
			priorCov = diag({1,2});
			postCov = priorCov;
			X = colvec(dq) || colvec(dpm);
			rc = BayesRegressionUpdate(priorMu, priorCov, dp, X, varu, postMu, postCov);
			if printLevel>=2 then print postMu postCov;
			coeffLower={0,-&Infinity};
			coeffUpper=j(2,1,&Infinity);
			coeffDraw = mvnrndT(postMu, postCov, coeffLower, coeffUpper);
			if printLevel>=2 then print coeffDraw;
			c = coeffDraw[1];
			beta = coeffDraw[2];
		end;

		if varuDraw then do;
			u = dp - c*dq - beta*dpm;
			priorAlpha = 1.e-12;
			priorBeta = 1.e-12;
			postAlpha = .;
			postBeta = .;
			rc = BayesVarianceUpdate(priorAlpha, priorBeta, u, postAlpha, postBeta);
			x = (1/postBeta) * rand('gamma',postAlpha);
			varu = 1/x;
			sdu = sqrt(varu);
			if printLevel>=2 then print varu;
		end;

		if qDraw then do;
			qDrawPrintLevel = 0;
			p2 = p - beta*pm;
			call qDraw(p2, q, c, varu, qDrawPrintLevel);
		end;
		
		parmOut[sweep, 1]=c;
		parmOut[sweep, 2] = beta;
		parmOut[sweep, 3] = varu;
		
end;
finish RollGibbsBeta;

Thank you again for your kind help!

Best regards,

Tammy

Re: ERROR: (execution) Module not loaded, operation not available

tammytt — Sun, 08 Aug 2021 16:13:53 GMT

Hi,

Thank you for your response!

I am running the code on the SAS of WRDS Cloud, as the code is based the data in WRDS. The SAS version there is

SYSVLONG=9.04.01M7P080520

I have tried to run it on the WRDS web SAS studio, but it just keeps running for like a full day and night without a final result, so I couldn't determine whether it used to work on the studio or not.

Each time I run the code, I would rerun the sas file which the ROLLGIBBSBETA module is in to get a new this.IMLSTOR file. And I assume this should works like "RESTORE", i am not sure though..

Re: ERROR: (execution) Module not loaded, operation not available

sbxkoenk — Sun, 08 Aug 2021 17:02:41 GMT

Hello @tammytt ,

You are running (or WRDS Cloud is running) SAS 9.4 Maintenance Level 7 (build date: 05AUG2020).

That's the last SAS 9.4 before SAS VIYA, so that's good! I mean, SAS is at a good level.

If you are running on the WRDS Cloud (I presume that is : Wharton Research Data Services - University of Pennsylvania), does it mean that this

RollGibbsBeta

IML-module was written over there? If yes, @mkeintz can maybe help you out. I believe (not sure though) he's working over there.

Spelling GAMMA or gamma as GAMMa is perfectly fine. SAS (the programming language, not the data sets) is case insensitive in general (except for folder paths and filenames if you are in a UNIX environment and a few other exceptions).

Defining (and compiling) your IML-module each time again should be an effective strategy to have it always available.
So, I do not know what goes wrong in your case.

Maybe the numeric shape parameter 'postalpha' in the RAND('GAMMA', postalpha) is becoming too big? I do not know about an upper limit for this shape parameter but I can imagine problems occur at values > 1E20. Not sure though. I'm just guessing.
Can you run it (the concerned module) on sample data or with a limited amount of real data (or a limited amount of loops)? Have you run it successfully before?

For the particulars of the STORE / RESTORE ( / LOAD ) functionality and commands for storing modules (and matrices) in SAS/IML, I think @Rick_SAS is better placed to answer that.

Good luck,
Koen

Re: ERROR: (execution) Module not loaded, operation not available

tammytt — Mon, 09 Aug 2021 09:41:20 GMT

Hi Koen,

Thanks for your help!

I haven't run it successfully before. But I do get the codes of testing the subroutines, which works just fine as following.

*______________________________________________________________________________________________

	Test the routines
 ______________________________________________________________________________________________;
proc iml;
	start main;
	call streaminit(1234);
	reset storage=this.imlstor;
	load;

	print 'Test of RandStdNorm:';
	x =RandStdNormT(.5,1);
	print x;
	
	print 'Test of mvnrndT:';
	mu={1 2};
	cov={1 .5, .5 1};
	vLower = {0 0};
	vUpper = {&infinity &infinity};
	x = mvnrndT(mu, cov, vLower, vUpper);
	print x;
	
	print 'Test of Bayesian normal regression update';
	nObs = 1000;
	nv = 2;
	x = j(nObs,nv);
	u = j(nObs,1);
	sdu = 2;
	do t=1 to nObs;
		x[t,1] = 1;
		u = sdu*rand('normal');
		do i=2 to nv;
			x[t,i] = rand('normal');
		end;
	end;
	y = x * colvec((1+(1:nv))) + u;
	priorMu={0,0};
	priorCov={&Infinity 0, 0 &Infinity};
	dVar = sdu*sdu;
	postMu = 0;
	postCov = 0;
	rc = BayesRegressionUpdate(priorMu, priorCov, y, X, dVar, postMu, postCov);
	print postMu;
	print postCov;
	priorAlpha = 1.e-6;
	priorBeta = 1.e-6;
	postAlpha = 0;
	postBeta = 0;
	u = y - X*postMu;
	rc = BayesVarianceUpdate(priorAlpha, priorBeta, u, postAlpha, postBeta);
	print postAlpha;
	print postBeta;
	finish main;
	run;
	quit;
	
proc iml;
	start main;
	call streaminit(1234);
	reset storage=this.imlstor;
	load;

	nObs = 250;
	sdm = sqrt(0.3##2 / 250);
	sdu = sqrt(0.2##2 / 250);
	print sdu;
	u = j(nObs,1,.);
	v = j(nObs,1,.);
	call randseed(12345);
	call randgen(u,'normal',0,sdu);
	call randgen(v,'normal',0,sdm);
	beta = 1.1;
	c = .04;
	q = j(nObs,1,.);
	call randgen(q,'uniform');
	q = sign(q-0.5);
	probZero = 0.6;
	z = j(nObs,1,.);
	call randgen(z,'uniform');
	q = (z>probZero)#q;
	p = j(nObs,1,.);
	m = 0;
	do t=1 to nObs;
		m = m + beta*v[t] + u[t];
		p[t] = m + c*q[t];
	end;
	pM = t(cusum(t(v)));
	
	nSweeps = 1000;
	regDraw = 1;
	varuDraw = 1;
	qDraw = 1;
	nDrop = 200;
	
	call RollGibbsBeta(parmOut, p, pm, q, nSweeps, regDraw, varuDraw, qDraw, 0,0,0,0);
	
	p2 = parmOut[(nDrop+1):nSweeps,];
	p2 = p2 || sqrt(p2[,3]);
	pm = p2[+,]/(nSweeps-nDrop);
	print pm;


	finish main;
run;
quit;

Re: ERROR: (execution) Module not loaded, operation not available

tammytt — Mon, 09 Aug 2021 09:53:51 GMT

And another thing is because I have run the program for several times, in each new run I just created a new folder and copy my two .sas files to the this new directory (the one with the subroutine and the main one). Could that be a problem like chaos with old one?

I have pasted here the contents of the subroutine file, any hints will be greatly appreciated!

options nodate nocenter nonumber ps=70 ls=120 nomprint; 
libname this '.';  

%let Infinity=1e30;
%let eps=1e-30;

*______________________________________________________________________________________________

	Define subroutines that will be used in simulations.
 ______________________________________________________________________________________________;
proc iml;

start main;
reset storage=this.imlstor;
store module=_all_;
remove module=main;
show storage;
finish main;


*____________________________________________________________________________________________________

	RollGibbsBeta: Estimate Roll model with Gibbs sampler
	
	The return argument is parmOut[nSweeps,3]
	Column 1:	c
	Column 2: beta
	Column 3: varu
____________________________________________________________________________________________________;


%let Infinity=1e30;
%let eps=1e-30;

start RollGibbsBeta(parmOut, p, pm, q, nSweeps, regDraw, varuDraw, qDraw, varuStart, cStart, betaStart, printLevel);
	nObs = nrow(p);
	if nrow(q)^=nObs | nrow(pm)^=nObs then do;
		print 'RollGibbsBeta length mismatch';
		return;
	end;
	dp = p[2:nObs] - p[1:(nObs-1)];
	
	if qDraw then do; 	*	Initialize qs to price sign changes;
		qInitial = {1} // sign(dp);
		qInitial = qInitial # (q^=0);	*	Only initialize nonzero elements of q;
		q = qInitial;
	end;

	if varuStart<=0 then varuStart = 0.001;
	varu = varuStart;
	if cStart<=0 then cStart=0.01;
	c = cStart;
	if betaStart<=0 then betaStart=1;
	beta = betaStart;
	
	parmOut = j(nSweeps,3,.);

	do sweep=1 to nSweeps;

		dq =  q[2:nObs] - q[1:(nObs-1)];
		dpm = pm[2:nObs] - pm[1:(nObs-1)];

		if regDraw then do;
			priorMu = {0,1};
			postMu = priorMu;
			priorCov = diag({1,2});
			postCov = priorCov;
			X = colvec(dq) || colvec(dpm);
			rc = BayesRegressionUpdate(priorMu, priorCov, dp, X, varu, postMu, postCov);
			if printLevel>=2 then print postMu postCov;
			coeffLower={0,-&Infinity};
			coeffUpper=j(2,1,&Infinity);
			coeffDraw = mvnrndT(postMu, postCov, coeffLower, coeffUpper);
			if printLevel>=2 then print coeffDraw;
			c = coeffDraw[1];
			beta = coeffDraw[2];
		end;

		if varuDraw then do;
			u = dp - c*dq - beta*dpm;
			priorAlpha = 1.e-12;
			priorBeta = 1.e-12;
			postAlpha = .;
			postBeta = .;
			rc = BayesVarianceUpdate(priorAlpha, priorBeta, u, postAlpha, postBeta);
			x = (1/postBeta) * rand('gamma',postAlpha);
			varu = 1/x;
			sdu = sqrt(varu);
			if printLevel>=2 then print varu;
		end;

		if qDraw then do;
			qDrawPrintLevel = 0;
			p2 = p - beta*pm;
			call qDraw(p2, q, c, varu, qDrawPrintLevel);
		end;
		
		parmOut[sweep, 1]=c;
		parmOut[sweep, 2] = beta;
		parmOut[sweep, 3] = varu;
		
end;
finish RollGibbsBeta;


*______________________________________________________________________________________________

call qDraw(p, q, c, varu, printLevel)	makes new draws for q

parameters:
p		column vector of trade prices
q		column vector of qs (REPLACED ON RETURN)
c		cost parameter
varu	variance of disturbance
printLevel	used to generate debugging output

_______________________________________________________________________________________________;
start qDraw(p, q, c, varu, printLevel);
if nrow(p)^=nrow(q) then do;
	print "qDraw. p and q are of different lengths. p is " nrow(p) "x" ncol(p) "; q is " nrow(q) "x" ncol(q);
	abort;
end;
if nrow(c)^=1 then do;
	print "qDraw. c should be a scalar. It is " nrow(c) "x" ncol(c);
	abort;
end;
if nrow(varu)^=1 then do;
	print "qDraw. varu should be a scalar. It is " nrow(varu) "x" ncol(varu);
	abort;
end;
if printlevel>0 then print p q;
qNonzero = q^=0;
q = q || q;
p2 = p || p;
nSkip = 2;
modp = colvec( mod(1:nrow(p),nSkip) );
ru = uniform(j(nrow(p),1,0));
do iStart=0 to (nSkip-1);
	if printLevel>0 then print iStart [l="" r="qDraw. iStart:" f=1.];
	k = modp=iStart;
	jnz = loc( k & qNonzero );	*	These are the q's we'll be drawing.;
	if printLevel>0 then print jnz [l="Drawing q's for t=" f=3.];
	q[jnz,1] = 1;
	q[jnz,2] = -1;
	cq = c*q;
	v = p2 - cq;
	u = v[2:nrow(v),] - v[1:(nrow(v)-1),];
	if printLevel>0 then print u [l="u:"];
	s=(u##2)/(2*varu);
	if printLevel>0 then print s [l="s"];
	sSum = (s//{0 0}) + ({0 0}//s);
	if printLevel>0 then print sSum [l="sSum (before reduction)"];
	sSum = sSum[jnz,];
	if printLevel>0 then print sSum [l="sSum (after reduction)"];
	logOdds = sSum[,2] - sSum[,1];
	*	Make sure that we won't get overflow when we call exp();
	logOkay = logOdds<500;
	Odds = exp(logOkay#logOdds);
	pBuy = Odds/(1+Odds);
	pBuy = logOkay#pBuy + ^logOkay;
	if printLevel>0 then print pBuy [f=e10.];
	qknz = 1 - 2*(ru[jnz]>pBuy);
	q[jnz,1] = qknz;
	if istart<(nSkip-1) then q[jnz,2] = qknz;
end;
q = q[,1];
finish qDraw;
*______________________________________________________________________________________________

rc = BayesVarianceUpdate(priorAlpha, priorBeta, u, postAlpha, postBeta)
	updates the variance posterior (inverted gamma)
inputs:
priorAlpha and priorBeta
u	vector of estimated disturbances
postAlpha and postBeta are updated on return to the posterior values
_______________________________________________________________________________________________;
start BayesVarianceUpdate(priorAlpha, priorBeta, u, postAlpha, postBeta);
postAlpha = priorAlpha + nrow(u)/2;
postBeta = priorBeta + (u##2)[+]/2;
return (0);
finish BayesVarianceUpdate;

*______________________________________________________________________________________________

rc = BayesRegressionUpdate(priorMu, priorCov, y, X, dVar, postMu, postCov)
	computes coefficient posteriors for normal Bayesian regression model
inputs:
priorMu and priorCov	coefficient priors (mean and covariance)
y	column vector of l.h.s. values
X	matrix of r.h.s. variables
dVar	error variance (taken as given)
postMu and postCov 	coefficient posteriors.
rc is a return code (0 if okay)
_______________________________________________________________________________________________;
start BayesRegressionUpdate(priorMu, priorCov, y, X, dVar, postMu, postCov);
if ncol(priorMu)^=1 then do;
	print "BayesRegressionUpdate. priorMu is " nrow(priorMu) "x" ncol(priorMu) " (should be a column vector)";
	return(-1);
end;
if nrow(X)<ncol(X) then do;
	print "BayesRegressionUpdate. X is " nrow(X) "x" ncol(X);
	return (-1);
end;
if nrow(X)^=nrow(y) | ncol(y)^=1 then do;
	print "BayesRegressionUpdate. X is " nrow(X) "x" ncol(X) "; y is " nrow(y) "x" ncol(y);
	return (-1);
end;
if nrow(priorMu)^=ncol(X) then do;
	print "BayesRegressionUpdate. X is " nrow(X) "x" ncol(X) "; priorMu is " nrow(priorMu) "x" ncol(priorMu)
		" (not conformable)";
	return (-1);
end;
if nrow(priorCov)^=ncol(priorCov) | nrow(priorCov)^=nrow(priorMu) then do;
	print "BayesRegressionUpdate. priorMu is " nrow(X) "x" ncol(X) "; priorCov is " nrow(priorCov) "x" ncol(priorCov);
	return (-1);
end;

covi = inv(priorCov);
Di = (1/dVar)*(t(X)*X) + covi;
D = inv(Di);
dd = (1/dVar)*t(X)*y + covi*priorMu;
postMu = D*dd;
postCov = D;
return (0);
finish BayesRegressionUpdate;

*______________________________________________________________________________________________

RandStdNormT(zlow,zhigh) returns a random draw from the standard normal distribution
truncated to the range (zlow, zhigh).
_______________________________________________________________________________________________;
start RandStdNormT(zlow,zhigh);
if zlow=-&Infinity & zhigh=&Infinity then return (normal(seed));
PROBNLIMIT = 6;
if zlow>PROBNLIMIT & (zhigh=&Infinity | zhigh>PROBNLIMIT) then return (zlow+100*&eps);
if zhigh<-PROBNLIMIT & (zlow=-&Infinity | zlow<-PROBNLIMIT) then return (zhigh-100*&eps);
if zlow=-&Infinity then plow=0;
else plow = probnorm(zlow);
if zhigh=&Infinity then phigh=1;
else phigh = probnorm(zhigh);
p = plow + rand('uniform')*(phigh-plow);
if p=1 then return (zlow + 100*eps);
if p=0 then return (zhigh - 100*eps);
return (probit(p));
finish RandStdNormT;

*______________________________________________________________________________________________

mvnrndT(mu, cov, vLower, vUpper) returns a random draw (a vector) from a multivariate normal 
distribution with mean mu and covariance matrix cov, truncated to the range (vLower, vUpper).
vLower and vUpper are vectors (conformable with mu) that specify the upper and lower truncation 
points for each component. 
_______________________________________________________________________________________________;
start mvnrndT(mu, cov, vLower, vUpper);
f = t(root(cov));
n = nrow(mu)*ncol(mu);
eta = j(n,1,0);
low = (vLower[1]-mu[1])/f[1,1];
high = (vUpper[1]-mu[1])/f[1,1];
eta[1] = RandStdNormT(low,high);
do k=2 to n;
	etasum = f[k,1:(k-1)]*eta[1:(k-1)];
	low = (vLower[k]-mu[k]-etasum)/f[k,k];
	high = (vUpper[k]-mu[k]-etasum)/f[k,k];
	eta[k] = RandStdNormT(low,high);
end;
return (colvec(mu)+f*eta);
finish mvnrndT;

run;
quit;

Re: ERROR: (execution) Module not loaded, operation not available

Rick_SAS — Mon, 09 Aug 2021 10:01:04 GMT

Can you post the code for the RANDSTDNORMT module? That seems to be the module that is reporting an error.

Re: ERROR: (execution) Module not loaded, operation not available

Rick_SAS — Mon, 09 Aug 2021 11:15:43 GMT

Thanks for posting the module definitions.

This might be impossible to solve without the data and a complete log. I spent all morning looking at this, but I am unable to reproduce it on simulated data. You might need to contact SAS Technical Support and arrange to provide the data and the full log.

I have a theory, which I cannot test. I will assume that this program used to work, even though you have never gotten it to work. If this program runs on different data every day/week/month, it might be that the data has properties that the program can't handle. For example, the program contains the following logic:

242  use temp.dsf where (permno=thisPermno & year=thisYear & kSample=thisKSample);
243    read all var {p pm q} into x [colname=colx];

Because of the WHERE clause, the number of rows for X depends on the data. It is possible that X has only 1 or 2 rows, and that is causing the program to fail. For example, you posted a Test Program that works, but if you change the program so that the data has less than five rows, the program fails:

proc iml;
	start main;
	call streaminit(1234);
	reset storage=this.imlstor;
	load module=_all_;

       /* the program WORKS if NOBS>=5, but fails for small NOBS */
	nObs = 4;    /* <== change this line */

	sdm = sqrt(0.3##2 / 250);
	sdu = sqrt(0.2##2 / 250);
	print sdu;
	u = j(nObs,1,.);
	v = j(nObs,1,.);
	call randseed(12345);
	call randgen(u,'normal',0,sdu);
	call randgen(v,'normal',0,sdm);
	beta = 1.1;
	c = .04;
	q = j(nObs,1,.);
	call randgen(q,'uniform');
	q = sign(q-0.5);
	probZero = 0.6;
	z = j(nObs,1,.);
	call randgen(z,'uniform');
	q = (z>probZero)#q;
	p = j(nObs,1,.);
	m = 0;
	do t=1 to nObs;
		m = m + beta*v[t] + u[t];
		p[t] = m + c*q[t];
	end;
	pM = t(cusum(t(v)));
	
	nSweeps = 1000;
	regDraw = 1;
	varuDraw = 1;
	qDraw = 1;
	nDrop = 200;
	
	call RollGibbsBeta(parmOut, p, pm, q, nSweeps, regDraw, varuDraw, qDraw, 0,0,0,0);
	
	p2 = parmOut[(nDrop+1):nSweeps,];
	p2 = p2 || sqrt(p2[,3]);
	pm = p2[+,]/(nSweeps-nDrop);
	print pm;
	finish main;
   run MAIN;
quit;

The error is different from the one that you report, but the fact is that the program's behavior might depend on the data. So my question is: Can you run this program on "old" data? Does it only fail on "current" data?

If you can find one data set for which it works and another for which it fails, this Community or SAS Technical Support might be able to help you discover what properties of the data are causing it to fail.

Re: ERROR: (execution) Module not loaded, operation not available

tammytt — Mon, 09 Aug 2021 13:19:03 GMT

Thanks so much for your help!

Another information is although the module RollGibbsBeta didn't run till the end successfully, it did produce (a part of) "gibbsOut" file with 13962 rows and 7 columns, and the results seems quite appropriate with the output in c, beta, varu and sdu (at least looks like so). Could that mean the module RollGibbsBeta works fine at the beginning and some inappropriate data in the middle make it fail?

the gibbsOut output:

the main program:

*____________________________________________________________________________________________________

	This proc loops over all samples, calling the estimation routines.
_____________________________________________________________________________________________________;
options nonotes;
proc iml;
	start main;
	call streaminit(1234);
	reset storage=this.imlstor;	*	This contains the subroutines built in RollGibbsLibrary02.sas;
	load;

	reset printadv=1 log;
	use temp.pys;
	read all var {permno year kSample} where(nTradeDays>=60) into sample [colname=colSample];  *my correction;
	nSamples = nrow(sample);
	print 'nSamples=' nSamples;
	permno = sample[,1];
	year = sample[,2];
	kSample = sample[,3];
	outSet = j(1,7,.);
	varnames ={'permno','year','kSample','c','beta','varu','sdu'};
	create this.gibbsOut from outSet [colname=varnames];
	do iSample=1 to nSamples;
		thisPermno = permno[iSample];
		thisYear = year[iSample];
		thisKSample = kSample[iSample];
		if mod(iSample,500)=1 then do;
			t = time();
			ctime = putn(t,'time.');
			print ctime iSample '/' nSamples ': ' thisPermno thisYear thisKSample;
		end;
		if thisPermno>=&startPermno & thisYear>=1990 then do;
			use temp.dsf where (permno=thisPermno & year=thisYear & kSample=thisKSample);
			read all var {p pm q} into x [colname=colx];
			
			nSweeps = 1000;
			regDraw = 1;
			varuDraw = 1;
			qDraw = 1;
			nDrop = 200;
		
			call RollGibbsBeta(parmOut, x[,1],x[,2],x[,3], nSweeps, regDraw, varuDraw, qDraw, 0,0,0,0);
			
			p2 = parmOut[(nDrop+1):nSweeps,];
			p2 = p2 || sqrt(p2[,3]);
			pm = p2[+,]/(nSweeps-nDrop);
			outset = thisPermno || thisYear || thisKSample || pm;
			*print outset;
			setout this.gibbsOut;
			append from outSet;
		end;
	end;
	finish main;
run;
quit;
options notes;
proc print data=this.gibbsOut (obs=50);
run;

data this.crspGibbs2009;
	set this.gibbsOut;
run;

Re: ERROR: (execution) Module not loaded, operation not available

Rick_SAS — Mon, 09 Aug 2021 14:10:15 GMT

Yes. That is possible. This is good info to know.

Because this is an MCMC computation, it is also possible that the issue is caused by a random number that is extreme and is causing the program to misbehave. It would be interesting to change

call streaminit(1234);

call streaminit(4321);

and see if the error occurs at the same location (that is, the same number of rows processed until it fails). If the error occurs at the same place, it is likely the data that is causing the problem. If it occurs at a different place (or doesn't occur at all), it might be an extreme value for a random number.

Re: ERROR: (execution) Module not loaded, operation not available

tammytt — Tue, 17 Aug 2021 02:42:33 GMT

Hi,

I have spent my last week trying to find the problem. It turns out the change of streaminit(1234) doesn't help.

I tried to find the common problem when the loop breaks. The error occurs when the iSample = 24001, 45501, 72001,124001,194001... I have thought that it should be the problem of the data, but if I started another run from 24001 etc it can still run and break at another point.

And I have also found another interesting point that at each error the program outputs a gibbsout file with exactly 13962 rows of data. So I am wondering could it be some running limit of RAND function (each time the log points to the problem of rand) or something?

I pasted here the code of the main loop to make you see the picture.

*____________________________________________________________________________________________________

	This proc loops over all samples, calling the estimation routines.
_____________________________________________________________________________________________________;
options nonotes;
proc iml;
	start main;
	call streaminit(1234);  
	reset storage=this.imlstor;	*	This contains the subroutines built in RollGibbsLibrary02.sas;
	load;

	reset printadv=1 log;
	use temp.pys;
	read all var {permno year kSample} where(nTradeDays>=60) into sample [colname=colSample];  *my correction;
	nSamples = nrow(sample);
	print 'nSamples=' nSamples;
	permno = sample[,1];
	year = sample[,2];
	kSample = sample[,3];
	outSet = j(1,7,.);
	varnames ={'permno','year','kSample','c','beta','varu','sdu'};
	create this.gibbsOut from outSet [colname=varnames];
	do iSample=166001 to nSamples;
		thisPermno = permno[iSample];
		thisYear = year[iSample];
		thisKSample = kSample[iSample];
		if mod(iSample,500)=1 then do;
			t = time();
			ctime = putn(t,'time.');
			print ctime iSample '/' nSamples ': ' thisPermno thisYear thisKSample;
		end;
		if thisPermno>=&startPermno & thisYear>=1990 then do;
			use temp.dsf where (permno=thisPermno & year=thisYear & kSample=thisKSample);
			read all var {p pm q} into x [colname=colx];
			
			nSweeps = 1000;
			regDraw = 1;
			varuDraw = 1;
			qDraw = 1;
			nDrop = 200;
		
			call RollGibbsBeta(parmOut, x[,1],x[,2],x[,3], nSweeps, regDraw, varuDraw, qDraw, 0,0,0,0);
			
			p2 = parmOut[(nDrop+1):nSweeps,];
			p2 = p2 || sqrt(p2[,3]);
			pm = p2[+,]/(nSweeps-nDrop);
			outset = thisPermno || thisYear || thisKSample || pm;
			*print outset;
			setout this.gibbsOut;
			append from outSet;
		end;
	end;
	finish main;
run;
quit;

Thanks again for your help.

Best,

Tammy

Re: ERROR: (execution) Module not loaded, operation not available

Rick_SAS — Tue, 17 Aug 2021 12:34:10 GMT

> It turns out the change of streaminit(1234) doesn't help.

What does this mean? You changed that line to be STREAMINIT(4321) and what happened? Did the problem occur during the same iteration, and the gibbsout data set is exactly the same?

Since the problem did not occur on the simulated data that I created, I suspect we will not be able to diagnose this problem without knowing more about the data you are using. You should contact SAS Technical Support and work with them to provide the data. Alternatively, find data for which the problem occurs and that you can share with the Community.

Re: ERROR: (execution) Module not loaded, operation not available

HowiePand — Sun, 08 Oct 2023 04:55:50 GMT

Hi Tammy,

Have you solved the problem? I was recently using Hasbrouk's two packages to calculate equity liquidity, as you previously did. However, I met the same problem as you - "ERROR: (execution) Module not loaded, operation not available." and the loop stopped.

Could you please tell if how you fix the problem?

Best,

Howie