How to create a list of files in a network folder on another machine than the SAS EG61 server

[PoulRavn]

Hi, I hope I stated the problem in the title, but here are some details.

I run EG61 on Win7 against a remote SAS 9.3. When I had local SAS installed, I succesfully used a macro, written by R. deVenezia: FindFile, to created lists of files on other positions. However that macro does not work on EG61 for obvious reasons.

I browsed for other methods and found, http://www.mwsug.org/proceedings/2014/BB/MWSUG-2014-BB19.pdf, which gives examples, among others relevant for me, 3) Get a list of files. This methods uses a 'dirlist pipe' approach. I replace the folder locations with relevant data for my foldes, but cannot get it to run. I get this error:

"Insufficient authorization to access PIPE" & "Error in filename".

Does this error apply to the fact that I cannot use PIPE, or does it apply to the me not having the right access rights to the folder? I tried to access a local machine folder on c:, but it gives the same error.

I AM able to created SAS data sets and write other files to the folders, that I try to get file lists for, so it is not, that I generally have access limitations.

Any takes on what this might concern? And solutions?

Regards

Poul Ravn Sørensen

[Kurt_Bremser]

With the default parameters of a typical SAS two-tier installation, the workspace server runs with the noxcmd option, which precludes usage of filename pipe, the x statement, system() function and call system() subroutine.

Have your SAS admin change that in the workspace server metadata configuration.

[TomKari]

Are you sure your macro won't work if you run it on your remote server?

[PoulRavn]

Hi Tom, in reply to your suggestion, that the 'old' macro could still run on the server: No it cannot, due to:

"NOTE: Because NOXCMD or LOCKDOWN is in effect, the N option is turned on and no routines will be invoked."

So I suppose the same limitation applies for the macro as for the other methods, that I used.

Regards

Poul

[TomKari]

Yes, that makes sense.

The sample code in note that provides works very well...I've used it on a number of occasions. If you have difficulties, just post to the forum. I guarantee someone will be able to help you out.

Tom

[ronan · Accepted solution]

Kurt gives you the correct explanation : standard installation prevents you to run 'system commands' in your remote or even sometimes local SAS session. As regards remote session running on servers, take my advice : this is for your own good, please dont'try to remove this safety net. Like Casey Smith from SAS ( see https://support.sas.com/resources/papers/proceedings12/297-2012.pdf ), I recommend every SAS admin to just say no to shell access requests !

The paper you are quoting gives you the proper workaround, albeit for Unix/Linux only: SAS code ; see page 3, (2) File Functions :

You can try the sample code provided in this page : SAS Filesystem Toolbox - sasCommunity

[jakarman]

SAS did close the xcmd with 9.1 local but opened it with 9.3 again. http://blogs.sas.com/content/sasdummy/2011/09/01/xcmd-and-sas-9-3/
It is complete madness to block something where the user already has access to. Running with his credential and limited to data with his credential.

The same is true for serverside processing. It should be running with the used credentials and NOT with shared accounts.

The recommendation for admins, no not a recommendation but a mandatory guideline, is following the principles as set by eg ISO27002. ISO/IEC 27002 - Wikipedia, the free encyclopedia ISO/IEC 27002 code of practice That implies shell access should be allowed but implementing a OS security control layers is to be done.    

It is that mandatory guideline where Unix admins are failing to get compliant. Sadly enough SAS didn;t align with the global guidelines like ISO27002 instead of that did some things for hiding the complaints of Unix admins. If you would go to use R and get FDA instructions with those or working in financial world you should recognize this. 

[PoulRavn]

Hi all,

greatful for your thoughts!! Thanks.

The advice(s) for/against opening the xcmd access seems to hit some sore spots. Some advocate for, others against. So far, I succeeded in making my admin confused, as he did not quite understand the request. I will try to explain to him, what it is.

The examples in SAS Filesystem Toolbox - sasCommunity work; I can get example 5 to run quite easily. However the real interesting one, and the one resembling the functionality of FindFiles is example 6: Retrieving the filelist recursively. I can get this example to run as well, but it does not give me any attributes. Specifically I would like to get the size and the creation date of the enties. Is there any thing that I might add to the function, that would do this?

Thanks all for answers.

Poul

[Kurt_Bremser]

Well, if you look at example 6, think of the work you need to do in integrating the methods of example 5 to get the attributes, and compare that with running "find &path -ls" in a filename pipe and just reading the output, you can see why I consider the "noxcmd" crowd senseless. Especially when they use the strawman argument of security; your system is either secure (and will withstand any stupidity done with system exits) or not; if it is not, get rid of the incompetent "admin".

[ronan]

KurtBremser wrote:

; your system is either secure (and will withstand any stupidity done with system exits) or not; if it is not, get rid of the incompetent "admin".

Oversimplified. Shell access doesn't generally allow for fine-grained controls, so that the dichotomy 'secure' / 'not secure' is baseless : interactive shell sessions are by default unsecure because - my guess - this interface was originally designed for Unix administrators and not for everyone else, untrained. For instance, ask your Unix/Linux System admin how to prevent, for instance recursive deletes (rm -R) or permissions changes (chmod -R, umask etc.) in interactive shell sessions. If he's unable to provide a practical answer for all these issues, surely, that doesn't imply he's incompetent : he cannot be held responsible for all the unprevented misuse that untrained (un-checked) users can inflict on his system. Unix/Linux command line terminals, and perhaps Windows terminals as well are not playgrounds for non admin developers so don't expect anything good if the two populations intermingle somehow.

[Kurt_Bremser]

MY users cannot, I repeat, CANNOT, do damage to the system. Simply because they don't have the permissions necessary to do that. They can only damage their own data, and everything that was at least a day old is saved in a backup. I do not fear for the consequences of their actions; I much more fear myself as the system admin, because I do have the permissions necessary to cause havoc.

Once again, disabling xcmd for "security" reasons is just lying to yourself.

And the CLI of UNIX shells was designed for admins AND users.

The reluctance to work from the commandline and the desire to prevent users from doing that comes from the inherent non-security of the most widespread "operating system" in use, the ongoing catastrophe called MS Windows.

[jakarman]

Some text copied from the paper Ronan posted (Casey Smith): "As long as users have the necessary host operating system permissions, they will be able to directly access and read a SAS table. So, what this boils down to is that any real data security must be put in place at the physical host operating system layer."

From the same person (Casey Smith) http://support.sas.com/resources/papers/proceedings13/420-2013.pdf 

" WHAT IS *NOT* SECURE

The only thing worse than a lack of security is a false sense of security. There are a couple of areas in SAS Enterprise Guide that might look or feel like security, but should not be relied upon for security. It is important to be aware of their limitations"

Does it makes any sense to make things difficult to code with SAS routines eg deleting files but still being possible and than saying the same action would be dangerous using OS-commands? Of course not, having the right to do those actions it doesn't matter what tool is used. Let it be as easy and simple as possible.
When that action is not allowed you should implement that trustworthy that is implementing those OS-controls.

[jakarman]

@Ronan, the FDA Approval for using R is requiring fine grained controls at the OS-level and they are possible.
What is behind the problem of xcmd/noxcmd is lot of misunderstanding and lack of system-knowledge. Get your hands on the IT security-policies of your company as it must reference IES/ISO 27002:2013 (normally used financial regulator). When you found please give some details on the background why a shell should not be allowed. I have seen those and there is no reason for not allowing.

The problem  with Unix-Admins not allowing those OS-controls is coming from some ancient limitation as of not being possible of being a member of ore as 8 (POSIX) or 16 (Sun) groups. Windows is allowing that for 1000 groups and modern (64-bit) for 64K groups. Newer AIX versions did raise that limit from 128 (5.3) to 2048.  With a HFS and using sticky-bits sguid-bits  (directory-only) you have all to a implement a fine-grained implementation to be serviced for LDAP (end-users). Maybe Unix CLI is far better for admins AND users (agree Kurt). For a pitty with Windows the shell explorer access and self service (Excel) got very more acceptance. Being a blocker (noxcmd) and not an enabler at Unix environments helps MS (and Excel usage).    

Going into data-mining text-ming areas those kind of users are very capable of handling those approaches (commands). Why offending that kind of users thinking those high educated persons are nitwits?     
    

[ronan]

Jaap Karman wrote:

@Ronan, the FDA Approval for using R is requiring fine grained controls at the OS-level and they are possible.  

Could you, please, elaborate .How do you finely control 'system commands' at shell level ? OS-level is too broad to be meaningful in my opinion; for instance, fine-grain permissions at filesystem level are often possible but that's not the point.

Jaap Karman wrote:

What is behind the problem of xcmd/noxcmd is lot of misunderstanding and lack of system-knowledge. Get your hands on the IT security-policies of your company as it must reference IES/ISO 27002:2013 (normally used financial regulator). When you found please give some details on the background why a shell should not be allowed. I have seen those and there is no reason for not allowing.

That's your personal interpretation. Could you, please, tell me where in the ISO 27002 referential is it explicitly allowed to provide a remote shell acess to every user ? Compliance to an ISO standard is optional by definition in a free country so that it won't apply to every use case. And, please, once & for all , Jaap, don't try to transfer the burden of the evidence : if you argue that an ISO standard implies/dictates a shell access (why not ?) then prove it ! He who asserts must prove.

[jakarman]

Ronan, that is not my personal interpretation it is not a problem for a having shell access. In contrary it is your personal interpretation it is a problem and I asked you to prove it with underpinning that with your local business policies. As I know that information is "restricted" (CIA rating)  those cannot be published free. I have access to hose ISO/IEC 27k documents as you should also have. I only can say: nowhere is mentioned  shell access should be forbidden. The best fits are control 9.4.4 "The use of utility programs that might be capable of overriding system and application controls should be restricted and tightly controlled."  or 9.2.3 "The allocation and use of privileged access rights should be restricted and controlled."
As it not there what is the evidence? Conclusion: That is the evidence it is no problem to have shell-access to be allowed.

"He who asserts must prove." Marvelous you are stating it is a problem to allow. But only does that as a personal opinion without any independence reference. You are asking to prove an action is allowed, that is humbug. You must prove that it is NOT Allowed. Make a list of all your actions you do and than try to prove it is allowed as somebody would have written down that for you. That is an impossible kafkaesk approach  

For some fun 9.4.5 "Access to program source code should be restricted." As SAS is program source code you shouldn't use SAS at all. But be lucky it is guideline with objectives to be met. (a-g) where change control procedures is mentioned.

My reference for R:  http://www.r-project.org/doc/R-FDA.pdf note to pre-req for see 11.10 11.30 and how often the host system is refered to as security.

With NIST (US standard) see section 4.2.2 of  http://csrc.nist.gov/publications/nistpubs/800-123/SP800-123.pdf And please note what is there.
For German BSI: BSI-Standard 100-1: Information Security Management Systems (ISMS) ,  Dutch, SA SANZ and some ohters I found that ISO/IEC reference. 

A very  good text for understanding ISMS is coming from that BSI:

"9.1. Introduction

The descriptions of an information security management system have been kept very generic in this document and in the ISO standards 27000, 27001, and 27002 and only serve as a framework. In

practice, therefore, a great deal of freedom exists for the practical implementation of the generic specifications. The great challenge lies in establishing an ISMS in one's own institution that not only

helps to achieve the set security objectives but is also as cost-effective and economic as possible."

This is not my interpretation just copied that from the BSI, the same is in the ISO 27001.