X Commands in Batch Mode

[GreyJoy]

I think I am looking for confirmation of the problem that I have ran into:

I currently have a BAT. file that calls SAS and runs a program every month. The BAT file had commands in it that copied data from a server to the local drive for SAS to use while in batch mode. I thought i might be able to dynamically edit, move and delete things as necessary by taking the CMD lines in the BAT. file and using them in the SAS program via the X statement. 

On their own, if i were to open up SAS EG and run the program interactively everything works as intended. 

HOWEVER...

When I run the same program while I am not logged in via the BAT. file the program fails to run because no data is being read in.

My assumption is that : SAS can not touch the servers in batch mode, even through the X commands. I am basing this on the fact that the X command used to copy data from the server to the local drive did not work, but the X command used to create a new folder on the local did. 

Am I right in my trouble shooting or is there a work around to make my Windows command lines dynamic and work from the SAS code?

[Kurt_Bremser]

Are you copying from network shares, or using SFTP in the external commands?

[GreyJoy]

I am copying from, and attempting to save output to shared drives.
C:\ - Local
W:\- Shared
U:\- Shared

[Kurt_Bremser · Accepted solution]

Then I guess that the "batch mode" initiated by the scheduler does not mount the shares. This is something you should check with your Windows admins. Have you tried to use UNC addressing?

I am not Windows-savant enough for this by far.

[GreyJoy]

I’m not either but will look into it. Thank you for the suggestion

[SASKiwi]

I'm assuming here that SAS is running on your PC and when you use EG it is using SAS on your PC also (a local SAS server, not connection to a remote SAS server)? Is this correct? If it is all local and you have the mapped remote drives defined in Windows Explorer, then they should work regardless of whether they are in a BAT file or via a SAS X statement. Why don't you just stick to doing this in a BAT file - its probably tidier that way.

[GreyJoy]

The the reason for moving the X commands into the program is this: I am much more knowledgeable with SAS language then CMD or Powershell, so when I run these programs Files Paths, Folders and File Names are dynamically assigned (typically based on dates the program completed). The naming convention that I used can easily be fed into SAS macros that I input into a COPY or MOVE or use when I send out an email at the end of the SAS program. 

Essentially, I do not know how , or even if it can be done, to dynamically assign names of directories and files in the BAT. file. If i could then that would keep me from having to figure this out. 

[GreyJoy]

IT WORKS!!!

As i understand it, when logged out of the computer, SAS will not have access to the servers and must use only the resources available to it on the local drives. So I use the windows CMD lines in a batch file to pull data from the servers to the local for SAS to use, then more CMD lines in the batch file to copy and move the output back onto the shared servers. This worked but was not dynamic and required several slightly different programs for each report. 

Your suggestion of using the UNC paths is what i needed because the MAPPED drive names are not available when not logged in, SAS needs the actual name of the server to send to the windows command prompt via the X command in order for the actions to take place when a windows profile is not active in a working session. 

Thank you ALL for your help. I learned a lot. 

[Quentin]

I'm a bit confused by your setup.  

In your current BAT file, you have a command to copy data, and then a command to invoke SAS in batch mode.  Is that invoking a invoking a batch SAS session running on your PC?

When you schedule the current BAT file (via Windows Scheduler?), did you have to schedule it with options like "run with highest privileges" in order to get the network drives to mount?

When you move the copy command into the SAS program (running the COPY command via X statement) and run it via EG, and everything works as intended, is that EG session connected to a local SAS session running on your PC or is it connecting to a remote SAS server?

You say "when I run the same program while I am not logged in via the BAT file the program fails..."  At this point you have only a SAS program, how are you running it, just right-click and batch submit to a local SAS session?  What does 'not logged in via the BAT file' mean?  Does your original BAT file have some commands to log in to something?

The good news is your batch session has X command enabled.  So in a sense, this is not a SAS problem.  A SAS batch session can use an X command to run an OS command that does stuff on network drives. 

But you need to know more about the account that is running that batch session.  Is it running as you, or as some service account?  As @Kurt_Bremser mentioned, are the shared drives mounted?  Is it possible to run the batch session with more privileges (something like "run as administrator")?

Agree with Kurt's suggestion to try UNC paths.

Windows credential manager can allow you to connect to multiple network drives using different credentials for each one.  And when it works, it becomes transparent, because you just log in to windows as you and all the shared drives work via the stored credentials. But I don't know what sort of privileges a process needs in order to use that credential storage for authentication to network drives.

Big picture, what you want to do *should* be possible.  But I think troubleshooting may require more help from a windows admin type person.

[GreyJoy]

Im going to try to answer all your questions, hopefully this clears it up. 

SAS is located on my local machine. I do not remote in for my SAS sessions. I use BAT files that point to the SAS software and the program I want to run. Below is what the old BAT file text looks like:

ROBOCOPY U:\DataDump\ICECAP C:\Users\clevenge\Desktop\Grey\SAS_Batch\ICECAP\Batch_Programs\Data
"C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -lrecl max -sysin "C:\Users\clevenge\Desktop\Grey\SAS_Batch\ICECAP\Batch_Programs\Icecap_Temp_Graphs.sas"

The call for the SAS.exe and program file stays the same but i have removed the Robocopy in the BAT and replaced it with this section of code in the program:

%if %sysfunc(getoption(sysin)) ne %str() %then %do;
    %let fullname=%sysfunc(getoption(sysin));
	%let Auto_path=%substr(&fullname,1,%length(&fullname)-%length(%scan(&fullname,-2,%str(\/.)).%scan(&fullname,-1,%str(\/.)))-1)\data ;
	/*Location and File of Transpose SAS program Locally*/
	%let Trans_path=C:\Users\clevenge\Desktop\Grey\Macro Transpose.sas;
	x %unquote(%str(%'mkdir "&Auto_Path.\" %'));
	X %unquote(%str(%'xcopy U:\DataDump\ICECAP "&Auto_Path.\" /S /K /Q %'));
%end;

Where it goes awry is at the lasts X statement is that block. If i double click the BAT file that calls SAS.exe and the program everything runs normally and works. 

However, If i were to set the Task scheduler to run at 12:30 pm, go to the windows icon on my desktop and select "LOCK" and leave for lunch at 12:25 pm, I would return to find that the program ran but could not complete because the data was not copied from the U:/ server. 

The only difference between these two is whether I have entered my password to my computer to bring up my windows profile. The old BAT file worked perfectly, but the desire to move to this way is based on the ability to move, dynamically named files to shared locations so i don't have to remember to do it manually every time the program runs. 

X %unquote(%str(%'xcopy "C:\Users\clevenge\Desktop\Grey\SAS_Batch\ICECAP\Tempurature_log_graphs\Graphs_&Today.\Robert DSMB ICECAP TEMPERATURE LOG GRAPHS &Today..pdf" "U:\Projects\ICECAP\Programs\Greys_Code\Scheduled Programs\Tempurature Logs\" /q %'));

[Kurt_Bremser]

Windows does make a difference if a program is run by starting from the user interface or from the scheduler. Where is the mount of the network share to "drive" U: made? If you do it in Windows Explorer, then it's not done at the system level, but when the user in question logs in to the GUI. Since there is no GUI login from the scheduler - bang!

[GreyJoy]

I am not knowledgably enough about servers and windows to truly understand what you are saying. I do not know how the server "mounts". I would assume our Network administrator would be my person here that would know all this though. 

What I am taking from all this is that it may be a user permissions issue. 

[Sajid01]

Hello @GreyJoy 
With the information you have shared it is difficult to advice appropriately. From your statement it looks like you have a local SAS installation on your PC and your able to run X Commands locally.
About your statement

 I am basing this on the fact that the X command used to copy data from the server to the local drive did not work, but the X command used to create a new folder on the local did.

A user will be able to able to access data from the server by using UNC naming convention. For example if the file myfile.ext is on the drive named datadrive on the server name servername, then its UNC path will be \\servername\datadrive\myfile.ext

Here are some general observations on batch processes / X commands running on the server.

About X Commands. X commands can be run on the server as long as they are enabled. And they can run OS commands (Copy , create, delete files folder  etc...). This is done at many places routinely.
Now about batch commands running on the server. Typically batch processes are run on a server with a generic ID with appropriate permissions.  There is no need for the user to log in. Most business environment the ID's running batch process have necessary permissions to create /delete folders and file.

About folder paths. When a process runs on the server, the paths  are with reference to the server. For example  C:\ mean the C drive on the server. Typically shared drives are available on the server too.
Often in business environments batch process are not designed to read/write on the C drive of the server. There could be exceptions.
Discuss your problem with your SAS Administrator or your seniors, Hopefully you will have answers to your questions.

[GreyJoy]

Thank you for your response, I think you had some questions that a previous response had. 

Please look above at my lengthy response. I hope that I accurately conveyed my set up. I am not versed in network set up and IT language so I am trying to explain as best as I know how. 

Again, thank you for the in-depth comments and questions. This is really helping me learn a lot about what happens in the background and how SAS interacts with Windows.   

[Sajid01]

Thanks for the response. I see this in your response.

The call for the SAS.exe and program file stays the same but i have removed the Robocopy in the BAT and replaced it with this section of code in the program:

Just curious, was the macro code part of SAS program or .BAT file?