topic Re: How to manipulate files in DS2? in SAS Programming

How to manipulate files in DS2?

Jonas_Jing — Tue, 15 Jul 2025 10:00:52 GMT

I have an existing data step code to download a .sas7bdat table from http endpoint, and save it into the WORK library, and then read the data in it.

The code is like this:

filename cp_src filesrvc folderpath="/Public" name="hmeq_train.sas7bdat" recfm=N lrecl=32767; * Save the table to work.hmeq_train; filename cp_tgt "hmeq_train.sas7bdat" recfm=N lrecl=32767; data _null_; fcopy('cp_src','cp_tgt'); run; proc print data=work.hmeq_train; run;

But this is in single thread. I have tens of binary .sas7bdat files needs to be downloaded at the same. I noticed that DS2 can leverage multi-thread ability to run program concurrently. But I tried fcopy() or INFILE statement, they both don't work in DS2.

Is there a way to do the same thing in DS2?

Re: How to manipulate files in DS2?

yabwon — Tue, 15 Jul 2025 11:09:56 GMT

Since you are using filesrvc method, it looks like your data are on your Viya server, is that right?

If those files located in a single directory?

If yes you could use PROC COPY to get data from one location to another:

libname a ".../Public";
proc copy in=A out=work;
run;

If those files are in multiple locations you can create "combined" library, and also use PROC COPY:

libname a (".../Public1" ".../Publuic2" ".../Public3");
proc copy in=A out=work;
run;

Bart

Re: How to manipulate files in DS2?

Kurt_Bremser — Tue, 15 Jul 2025 11:12:19 GMT

This is not HTTP, this is the SAS Viya File Service
You would copy the file to the current working directory of your SAS process, which is (unless you change it explicitly in your code) different from WORK
Massive parallelization of I/O can easily lead to an overload of your storage subsystem, resulting in serious overall performance degradation, and no net gain

Re: How to manipulate files in DS2?

SASKiwi — Tue, 15 Jul 2025 20:52:15 GMT

@Jonas_Jing - I think you have an IO issue with copying multiple files not a multi-threading restriction. I believe you could use SAS/CONNECT, which comes with Viya, and start several SAS sessions and simulaneously download multiple files that way.