topic Re: Unlocking compiled macro library in SAS Programming

Unlocking compiled macro library

MattDSS — Fri, 12 Jun 2015 04:37:46 GMT

Compiled SAS macro libraries cannot be updated/overwritten while they are being read by one user. If I try to compile a macro while it is being read, I would get an error saying that "A lock is not available for LIBRARY.SASMACR"

If one of our user launched the library, but forgot to the turn it off before taking 3 days leave, then I can't change it for the next 3 days!

Is there anyway to override this and to give me privileges to the library?

Re: Unlocking compiled macro library

ballardw — Fri, 12 Jun 2015 15:54:36 GMT

Sounds like an IT sys admin question to log them off of a process.

Re: Unlocking compiled macro library

sbb — Fri, 12 Jun 2015 16:30:23 GMT

Cancel the user's session and tell the individual not to do it again - such serialization for any SAS OBJECT library is critical for successful operation.

Re: Unlocking compiled macro library

SASKiwi — Sat, 13 Jun 2015 02:26:22 GMT

To avoid this problem in the future you could re-design your use of compiled macros so each read-only user copies LIBRARY.SASMACR to their WORK directory and uses macros from there. Alternatively don't use pre-compiled macros, use AUTOCALL macros instead which get compiled on the fly to each user's WORK library. We use the latter approach and it avoids locking issues entirely.

Re: Unlocking compiled macro library

jakarman — Sat, 13 Jun 2015 05:56:15 GMT

The reason the read lock is needed is because of possible inconsistency when those compiled versions would change on the fly. That is a common logical situation happening in many more situations.

No, you cannot overwrite that protection.

The options:

Your release management process should have some Sla of time moments the updates could be done and allowing you to kill all blocking processes.

Work with local copies Eg a sas_work version for the persons doing too many updates (developers).

Re: Unlocking compiled macro library

MattDSS — Sat, 13 Jun 2015 07:55:09 GMT

Thanks for the suggestions everyone. So the answer is no. It would be good, in future SAS, to allow compilers of the macro to add a password or something that would give them privileges to the library.

For now, I can try the auto-call suggestion.

Re: Unlocking compiled macro library

jakarman — Sat, 13 Jun 2015 19:03:27 GMT

If you would think the fundamental logical problem behind locking is an Sas issue you are wrong go and look what is in acid with a rdbms system. That question of locking on artifacts that are getting updated is a generic one.

The Autocad is avoiding that question as the resulted compiled version is sa local one.

Re: Unlocking compiled macro library

MattDSS — Sun, 14 Jun 2015 23:15:31 GMT

Thanks for pointing the ACID thing.

I don't think it is a fundamental logical problem. I do think, though, that putting an special password on a file which would allow overwrite privileges to be a good idea.

Re: Unlocking compiled macro library

sbb — Sun, 14 Jun 2015 23:57:50 GMT

Any SAS OBJECT library would typically have similar "authorized" replacement consideration, whether it is a compiled-MACRO library, a SAS DATA library or a SAS FORMAT member.

It's imperative that rules/procedures be adopted to manage these environments, especially inadvertent or others undesired library "locking" situations.

Scott Barry

SBBWorks, Inc.

Re: Unlocking compiled macro library

jakarman — Mon, 15 Jun 2015 06:47:34 GMT

That ACID thing is coming from F.Codd father of SQL and got many rewards for that. The whole theory behind that is a logical one of who wants to update a object who are using that at the moment an how resolving those request conflicts.

As a logical process it is not solvable by means of some password. If our great awarded leaders came with this all supported by mathematics why wouldn't it be true. When you are convinced of that than your underpinning should be of same quality as they did.

Re: Unlocking compiled macro library

Kurt_Bremser — Mon, 15 Jun 2015 08:42:54 GMT

It is not a question of passwords etc. No operating system allows rewriting a complete file when a process has an open file handle on it. Only when certain random access modes are used can parts of files be updated while other processes also have access to the file.

A SAS catalog does not have a structure that lends itself easily to random access read/writes, so it is always rewritten completely.

So it is technically not possible to do what you want, at least not with justifiable effort.

Re: Unlocking compiled macro library

MattDSS — Mon, 15 Jun 2015 09:53:51 GMT

I think there is some misunderstanding about what I want with a "password". I do not want to write while others are reading - I understand this is not possible. Instead, I want to be able to abort users who are using my library.

Isn't it simple? If I enter a password when using my library, instead of returning "A lock is not available for LIBRARY" to me, it will return "Your program has been aborted because user X has taken control of LIBRARY." to whoever is occupying the catalog. Surely the IT admin can abort users. The question is whether this sort of abilities can be given to normal users.

On that note, I don't even know how users can disconnect THEMSELVES from the library. Whenever I tell my colleagues to return control of the library to me, nothing they try could work - setting nomstore, or clearing the libname. The only way for them to return control is to disconnect their profile. This is really inconvenient because they lose whatever they are working on.

Is there anyway they can disconnect themselves from the library? Or is there some fundamental reason in SAS that would prohibit this?

Re: Unlocking compiled macro library

Patrick — Mon, 15 Jun 2015 10:29:25 GMT

This is about a stable environment. If you would "abort users" then this is the same than users closing their sessions - they will loose their environment. If you have admin rights then you can kill sessions of other users - but it's quite "brutal". You can also simply re-start the object spawner which will end all sessions (except for batch processes).

What's the reason that you have to pre-compile macros? That's something I would only do in a serious production environment where you have proper release management - and then it's no problem as during the few release windows a year you can inform users that the server will be down so you won't have locking issues.

If this is more of an "adhoc reporting" environment where you're maintaining the "SAS framework" then I don't see any reason why you can't use the Autocall facility where you store macros as .sas files which then will get compiled and copied into the users work library once they first call the macro. Such an approach doesn't make any difference to users but you will never have locking issues.

Re: Unlocking compiled macro library

MattDSS — Mon, 15 Jun 2015 11:04:32 GMT

Aborting programs is different from aborting sessions right? A %abort, for example, doesn't cause users to lose their environment.

I am, actually, in the environment you are talking about. I could use the autocall, or I could, like SASkiwi suggests, get users to copy the library. This would work too.

Sorry for all the dumb questions! I am a newbie to SAS. I did some C++ when I did my doctorate in mathematics, but I only wrote for myself and didn't have these issues I am facing now.

Re: Unlocking compiled macro library

Patrick — Mon, 15 Jun 2015 11:40:33 GMT

Not really. During SAS invocation the environment gets established - the work library, SAS macro variables etc. etc. That's the same for a batch process than for a session.

So if you use SAS EG for example and you connect to a SAS Server a new SAS session gets "spawned" (using the SAS Object spawner). That's when all the config files and autoexec's get executed. If you end the session (or a batch process) using %abort then the session/process ends and you loose the environment (like the work space) - in a Windows environment you would see a sas.exe task terminate.

If you want to get a better understanding of how SAS works then I strongly recommend you read the SAS Concepts manual http://support.sas.com/documentation/cdl/en/lrcon/67885/PDF/default/lrcon.pdf (it's a very good and well written manual).

For your case: I strongly recommend you don't pre-compile macros but store them simply as .sas file in a folder which is part of the macro search path SAS(R) 9.4 Macro Language: Reference, Third Edition

You simply store your SAS macros in .sas files which have the name of the SAS macro (in UNIX: name must be all in lower case!). The folder location where you store these macros must be defined in SASAUTOS (that's simply the definition of a search path for macros).

When users call a macro then SAS will search for a .sas file in the macro search path (SASAUTOS). Part of this search path is Out of the box - and it can be an actual physical path or a catalog in a library. One of the first places where SAS looks for a macro is in the WORK library (for members in catalog SASMACR).

Sooo... You have your .sas file with the name of the macro stored in a folder which has been defined in SASAUTOS. The first time the user calls the macro in the code SAS will scan through the macro search path, find this .sas file, compile it, and store it in the macro catalog in WORK.

The second time the user calls the macro (in the same SAS session) SAS will again scan through the macro catalog. But it looks first in WORK, now finds the compiled macro there and will use this one (...so if you changed in-between the macro code in the .sas file the user will still get the old version).

Using SAS EG if the user disconnects from the Server the work-space gets cleaned out. When he re-connects a new work space gets created. So now if the user runs code calling the macro then SAS will again scan through the macro search path, there won't be a compiled macro in WORK so it will find it the first time as .sas file in the folder which is part of the search path... and the story starts from the beginning.

It's very long ago that I've done something with C++. Working with SAS is very different. Forget about compilation. With SAS it's all done "on the fly" during run-time. There are only a few exception to this - and as I understand the environment you're in you won't have to care about these exceptions. So: Don't care about compilation at all.

...Hope this made it clearer to you. Read the Concepts Manual - it's all in there and much much more - and it's may-be the most important SAS manual to be read if you want to get a deeper understanding of how SAS works.

Re: Unlocking compiled macro library

MattDSS — Mon, 15 Jun 2015 12:05:38 GMT

Thanks Patrick. The manual looks helpful.

I saw autocalls before and know about saving each macro as one file. This felt very strange to me coming from Python/C++ background. I feel my %start_timer %end_timer macros should be in one file in one folder and my %start_parallelisation %end_parallelisation macros should be in one file in another etc. If I have a large number of macros, and they all have their own files in one place.... then it will go very chaotic. Maybe I just haven't gotten use to the concept.

However, I know that autocall can read macro definitions from a catalog. I haven't gotten that to work because the SAS documentation didn't give much explanation for this. I am still trying to work it out.

Re: Unlocking compiled macro library

Patrick — Mon, 15 Jun 2015 12:14:38 GMT

It's about "modularization" - but you can call macros in macros if that makes sense (I normally don't do it as I try to keep things simple).

ma.sas

%macro ma();

....

%mend;

mb.sas

%macro mb();

....

%mend;

m_comb.sas

%macro m_comb();

%ma();

%mb();

%mend;

You could have these 3 .sas files all in the same folder (part of Autocall). If a user calls "m_comb()" then SAS will work out "on the fly" that it first has to compile the macros called within macro "%m_comb()".

Re: Unlocking compiled macro library

MattDSS — Mon, 15 Jun 2015 13:57:42 GMT

Thanks everyone for all their suggestions.!

Re: Unlocking compiled macro library

Ron_MacroMaven — Mon, 15 Jun 2015 14:20:36 GMT

the autocall and compiled features are different.

* autocall means that SAS search for a macro definition reference in a set of filerefs (folders) which are the value of the option sasautos

the autocall is turned on my default

%put mautosource:%sysfunc(getoption(mautosource));*is boolean;

%put %sysfunc(getoption(sasautos,keyword));* is character;

when a macro is called SAS searched the filerefs for a file with the name of the macro

and %includes it, the result of which is that the macro is compiled and stored in the work.sasmacr catalog.

* compiled and stored

(stored and compiled: not alphabetical, nor the sequence of events, but ...)

macros have the %macro statement option "/store" and when they are submitted, or %included,

then they are compiled and written not to work.sasmacr catalog

but to the libref which is the value of the option sasmstore.

%put mstored:%sysfunc(getoption(mstored));*is boolean;

%put %sysfunc(getoption(sasmstore,keyword));*is character

I kind of like the idea of copying the compiled and stored catalog to the user's work fileref

but that is a solution for multiple users in multiple sessions.

I think that is a lot of copying

another solution is to reset the libref of sasmstore.

it does not have to be 'library'.

note: files with macros with the store option set ought not to be in any of the filerefs in the sasautos search list.