@David_Billa wrote:
@Patrick any simple example to bulk load to SAS?
Not sure about Snowflake but with Redshift I found it faster to use their COPY command that could create multiple delimited text files to an S3 bucket. This allowed the database to generate the files very quickly. I could then copy the files to some place that was local to the SAS server and use a simple data step to read them in.
The code to read in a series of delimited text files when you have an example of the known structure is very simple. Define the variables (in the right ORDER!), read the list of filenames, for each filename read the observations from that file.
So if you have an existing dataset with the variables defined properly, say it is named TEMPLATE, and a dataset named FILELIST with a variable named FILENAME
data want;
if 0 then set template;
set filelist;
do while(not eof);
infile dummy filevar=filename dsd truncover firstobs=1 end=eof;
input (_all_) (+0);
output;
end;
run;
Here is an example you can run to see how it works in action:
filename csv temp;
data _null_;
set sashelp.class ;
file csv dsd ;
put (_all_) (+0);
run;
data filelist;
filename=pathname('csv');
run;
data want;
if 0 then set sashelp.class;
set filelist;
infile dummy filevar=filename dsd truncover firstobs=1 end=eof;
do while (not eof);
input (_all_) (+0);
output;
end;
run;
proc print;
run;
