BookmarkSubscribeRSS Feed
lmtamina
Obsidian | Level 7

Good Afternoon,

 

What is the recommended solution to export a SAS dataset  with 14Million observations and 55 variables  . We need to send claims details to our vendor on their FTP site. I am a beginner in SAS , and I have not had any experience exporting a large dataset . I am  currently  familiar with exporting dataset into Excel and CSV format. Appreciate any suggestions. Thank you as always.

6 REPLIES 6
ballardw
Super User

My first idea would be Proc Export to CSV file. But that would require the order of the variables to match a column order if there is a specified order that is expected on the FTP site. File size shouldn't matter to CSV as there is no number of records limit imposed by the file type and 55 variables isn't a large number and shouldn't normally have issues.

lmtamina
Obsidian | Level 7

Thank you both. I am not familiar with writing to filename zip with a delimiter. Is it possible to give an example or the syntax for this procedure. Will greatly appreciate it!

Kurt_Bremser
Super User

See this quick example for sashelp.class:

filename outfile zip "$HOME/sascommunity/class.zip" member="class.csv";

data _null_;
file outfile dlm=',';
set sashelp.class;
if _n_ = 1 then put 'name,sex,age,height,weight';
put
  name
  sex
  age
  height
  weight
;
run;

@lmtamina wrote:

Thank you both. I am not familiar with writing to filename zip with a delimiter. Is it possible to give an example or the syntax for this procedure. Will greatly appreciate it!


 

Reeza
Super User

I just downloaded and imported a 86 million row CSV. 

 

CSV will work but you'll need to zip it and it'll still be big. 

 

You can consider splitting your files into multiple subsets and then sending them piecemeal. 

Not ideal but probably the easiest way to do this.

 

Here's an example on how to split it into small subsets by size

https://gist.github.com/statgeek/f4eaec3e5ec9666865e663c9e225f09d

 

mnjtrana
Pyrite | Level 9
I think importing to csv will get your job done, but your file will be huge, so if you use unix, you zip the file using gz command or use the filename zip statement

Cheers from India!

Manjeet

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 6 replies
  • 7005 views
  • 3 likes
  • 5 in conversation