BookmarkSubscribeRSS Feed
pjoice
Calcite | Level 5

I have delimited txt files that I want to import into SAS. Each contains character and numeric variables, some of which are more interesting than others. Each file contains millions of records, so importing all of the cases and all of the variables is taking way too long.

I'm currently running a proc import (sample code below), but it's taking forever. Is there a more efficient way to do this? Can the proc import specify certain variables to drop before even reading in the program datavector? For instance, if I only want to import var1 and var3, can I specify that in the proc import?

sample code:

proc import datafile="C:\rawdata\sales.txt" out=output.sales dmbs=dlm;

     delimiter="|";

     getnames=yes;

run;

data sales_small;

     set output.sales (keep= var1 var3);

run;

2 REPLIES 2
PGStats
Opal | Level 21

You could put the dataset options in the proc import statement :

proc import datafile="C:\rawdata\sales.txt" out=output.sales(keep=var1 var3) dmbs=dlm;

PG

PG
data_null__
Jade | Level 19

If your CSVs have a field name row as the first record and IF you know

  1. The name of the field(s) you want to read.
  2. The data type
  3. The informat

You can use a program like that described here http://www.lexjansen.com/pharmasug/2011/TT/PharmaSUG-2011-TT04.pdf to read the files.  If you have many files you can read them all at once, gathering the fields of interest from each.

If you don't have the field name row you could still use a similar technique but it would be simplified by just specifying the field relative positions.  You will still need to know the type and informat.

You won't actually need to specify informat for most fields as most character and numeric fields will be read with the SAS default.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 2 replies
  • 615 views
  • 0 likes
  • 3 in conversation