Good Afternoon,
What is the recommended solution to export a SAS dataset with 14Million observations and 55 variables . We need to send claims details to our vendor on their FTP site. I am a beginner in SAS , and I have not had any experience exporting a large dataset . I am currently familiar with exporting dataset into Excel and CSV format. Appreciate any suggestions. Thank you as always.
My first idea would be Proc Export to CSV file. But that would require the order of the variables to match a column order if there is a specified order that is expected on the FTP site. File size shouldn't matter to CSV as there is no number of records limit imposed by the file type and 55 variables isn't a large number and shouldn't normally have issues.
Write to filename zip with a delimiter. It's probably the most compact result.
Thank you both. I am not familiar with writing to filename zip with a delimiter. Is it possible to give an example or the syntax for this procedure. Will greatly appreciate it!
See this quick example for sashelp.class:
filename outfile zip "$HOME/sascommunity/class.zip" member="class.csv";
data _null_;
file outfile dlm=',';
set sashelp.class;
if _n_ = 1 then put 'name,sex,age,height,weight';
put
name
sex
age
height
weight
;
run;
@lmtamina wrote:
Thank you both. I am not familiar with writing to filename zip with a delimiter. Is it possible to give an example or the syntax for this procedure. Will greatly appreciate it!
I just downloaded and imported a 86 million row CSV.
CSV will work but you'll need to zip it and it'll still be big.
You can consider splitting your files into multiple subsets and then sending them piecemeal.
Not ideal but probably the easiest way to do this.
Here's an example on how to split it into small subsets by size
https://gist.github.com/statgeek/f4eaec3e5ec9666865e663c9e225f09d
Available on demand!
Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.