BookmarkSubscribeRSS Feed
marleeakerson
Calcite | Level 5

Hello, 

 

I am having trouble importing a very large TSV file into sas (>1million KB). Does anyone know how to import a file of this size? 

 

Thank you. 

6 REPLIES 6
Kurt_Bremser
Super User

Importing tab-delimited files with a data step is rather simple. What is your problem?

In case of ERRORs, please post the log, using the {i} button.

marleeakerson
Calcite | Level 5

I dont know if the file is too big or what, but for some reason when i upload the dataset into sas a lot of the columns are concatenated and condensed. here is my code:

 

proc import datafile="X:\Temp\ABC\ARCOS\arcos-oh-statewide-itemized.tsv" out=work.Oh dbms=tsv replace;

ballardw
Super User

Try DBMS=TAB instead of DBMS=TSV.

 

also you might want to add the statement:

Guessingrows=32000;

 

So more rows of data are examined before the variable types, lengths and informats are set by the procedure.

Reeza
Super User
You can import text files uisng proc import or a data step. In a data step you have to specify the formats and types while PROC IMPORT does that but it has to guess. PROC IMPORT is slow because it has to guess the data. Instead, take the code from the log after PROC IMPORT, adjust it as necessary and run the data step instead. That's much faster at importing data because it's no longer guessing.
PGStats
Opal | Level 21

@marleeakerson wrote:

a lot of the columns are concatenated and condensed.


Make sure the columns are effectively separated by tabs.

PG

sas-innovate-2024.png

Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

 

Register now!

Mastering the WHERE Clause in PROC SQL

SAS' Charu Shankar shares her PROC SQL expertise by showing you how to master the WHERE clause using real winter weather data.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 6 replies
  • 8555 views
  • 8 likes
  • 5 in conversation