Copy different files to different local locations while using SAS eg on server

[Kees]

Allright, so I have a project to create a lot of datasets and those datasets have to go to a lot of different locations. Each dataset has his own interval (once a week, once a day, once a month, etc). At the same time, all of this is and will stay under development, meaning every now and then there will be new datasets to copy to new locations.

We use SAS eg on a linux server environment and till now, I use the "Copy Files" task to get datasets to our windows environment. This means one Copy Files for every dataset-location combination.

My goal is to not have to use those specific "Copy Files" tasks. I don't want to manually have to create a Copy Files everytime my collegues make a new dataset. I want to ask them to write the dataset name in a spreadsheet, write down the location of the spreadsheet and I want the Copy Files funtionality to be able to use this input and export all the files.

I've tried working with proc export, this is limit to the Linux environment only so this is not an option. I thinks assigning a library is also not going to work because of the server. I also tried giving a list of files and locations to the Copy Files but couldn't get that to work.

Any help would be appreciated!

[LinusH]

Hi, can you be bit more specific with the limitations for PROC EXPORT in Linux?

What file formats do you need to support?

Where are you sending/copying the files?

I assume you don't have a license for PC File Formats (that would give possibility to create "Windows" files using a PC Files Server on Linux.

Another option would be to have the users get the data themselves, in a mote centralized set up. But then you need SAS/SHARE and the users need to use SAS ODBC driver.

[Kees]

Thanks for answering.

I need to create xlsx files from the tables. I use the proc export to do this.

But the proc export can only reach inside our linux environment.

So if I try this, it works:

PROC Export
DATA=WORK.testdata
   OUTFILE="~/SASDatagebied/DG_I_40ANA_X/test/test.xlsx"
   DBMS=XLSX Replace;
RUN;

And this doesn't:

PROC Export
DATA=WORK.testdata
OUTFILE="Q:/test/test.xlsx"
DBMS=XLSX Replace;
RUN;

Because now it will search for "~/SASDatagebied/user/Q:/test/test.xlsx"

So, hence my conclusion, the proc export can't reach without the server environment. And I need my files outside.

[Reeza]

---

@Kees wrote:

Thanks for answering.

I need to create xlsx files from the tables. I use the proc export to do this.

But the proc export can only reach inside our linux environment.

---

That is correct, EG is a server based tool, so think like the cloud. It can't see your computer unless you explicitly set it up to see that location. So you'll need to have an administrator set it up so EG can see each location you need to export to. 

A semi workaround is to export to a central local location and have a local job that moves the files to where they need to be, which is easier to manage. That will likely be a Powershell or DOS job unless you have SAS also running locally.  

[Kees]

I work in a large organisation with really high security standards. So my SAS or IT administrators won't set something up just for me. If I really want to walk this road it will literally take me years and I don't have that time.

Export to a central location and a local job to move the files around is a possibility indeed, I considered this myself but I would prefer something in SAS itself because I can schedule SAS on a server but I can't schedule anything else outside my local laptop and I don't want to be depend on my own laptop.

The Copy Files task can export files from the SAS linux environment to the Windows environment so I'm looking for something there.

Is it possible to give an array of files and an array of locations to a Copy Files task?

Or is it possible to build a loop around a Copy Files task?

[Reeza]

Last time I checked Copy Tasks couldn't even be included as part of a process flow to be automated, has that changed?

[Kees]

That would be really annoying. We're still in the process of getting acces to SAS Management Console and everything else needed to schedule so for the moment I just schedule with windows taskplanner and a SAS flow named AUTO EXEC.

[Kurt_Bremser]

An organization striving for high security standards MUST INSIST that all steps leading to data involved in business decisions are properly documented. This means batch processing of all SAS codes which create datasets, where all logs from every execution are kept. This in turn means that all resources needed are available on the server, as a client tool like EG simply cannot meet these requirements.

This must be made clear to those who design the overall process.

[SASKiwi]

Folder shares can be created on linux to point to other servers, including Windows ones. You need to engage with your SAS or IT administrators to set that up. Once that is done PROC EXPORT should work fine. 

[Sajid01]

Hello @Kees 
There are multiple ways to achieve this but it depends on your local environment.

1, If the windows location is accessible by using a UMC path for example \\your_Window_host\full_full_path.
  See if you can create a libname using this. If this works you can move forward. I have used this in one of the place I worked.

libname mylib "\windows_host\desired_path";

2.Create a batch script or Powershell script on the Windows system to copy the file(s) from SCP files from the Linux machine to Windows machine. 
Present versions of windows come with openSSH OR you can use pscp (this is a free scp tool).
I have batch processes which use this approach to download files from Linux to Windows., managed by scheduler with parent process as the predecessor / dependency.
If you like you can seek assistance from  Windows / Linux Admins or may be SAS Admin. The exact details depend upon your requirement.
In your situation I would use this approach.
There is a SAS note on SSH that can be helpful. https://support.sas.com/content/dam/SAS/support/en/technical-papers/configuring-ssh-client-software.... 
3.Lastly the any FTP tool like FilZilla. These are manual steps but you may not like but are helpful.

[Kees]

I will probably persue something in line with your second solution. The first sadly doesn't work.

Thanks for the info!

[Patrick]

You write your company has a tight security policy. You probably want to ensure that you're not breaching any such policies by transferring data off the server environment - especially if it's PII/sensitive data. 

You mentioned management console so I guess this will in the end be a fully server side scheduled process that runs in batch without any connection to the client side. 

One simple way for distribution of not too high data volumes is to have SAS send emails with attachments.

[SASKiwi]

Sounds like you are working in a Windows environment. So do we and we use UNC paths all the time to read from one server to another. They are incredibly useful. For example on our PCs in Windows Explorer, we can type the UNC paths for all of our SAS server drives and view all files. Often getting UNCs working requires both folder permissions and Kerberos settings changing. An IT administrator needs to do this.

I suggest you talk to your SAS administrator about your requirements and what are the best options.