Importing large TSV files into SAS

[marleeakerson]

Hello, 

I am having trouble importing a very large TSV file into sas (>1million KB). Does anyone know how to import a file of this size? 

Thank you. 

[Kurt_Bremser]

Importing tab-delimited files with a data step is rather simple. What is your problem?

In case of ERRORs, please post the log, using the {i} button.

[marleeakerson]

I dont know if the file is too big or what, but for some reason when i upload the dataset into sas a lot of the columns are concatenated and condensed. here is my code:

proc import datafile="X:\Temp\ABC\ARCOS\arcos-oh-statewide-itemized.tsv" out=work.Oh dbms=tsv replace;

[ballardw]

Try DBMS=TAB instead of DBMS=TSV.

also you might want to add the statement:

Guessingrows=32000;

So more rows of data are examined before the variable types, lengths and informats are set by the procedure.

[Reeza]

You can import text files uisng proc import or a data step. In a data step you have to specify the formats and types while PROC IMPORT does that but it has to guess. PROC IMPORT is slow because it has to guess the data. Instead, take the code from the log after PROC IMPORT, adjust it as necessary and run the data step instead. That's much faster at importing data because it's no longer guessing.

[Kurt_Bremser]

Don't use import. Write a data step according to the documentation you got with the file. You can use the code created by proc import as a starting point.

[PGStats]

---

@marleeakerson wrote:

a lot of the columns are concatenated and condensed.

---

Make sure the columns are effectively separated by tabs.