topic Re: Loading Huge Dataset in SAS Data Management

Loading Huge Dataset

ilya_1991 — Fri, 27 Nov 2015 15:26:53 GMT

Good Day.
I am trying to upload txt file with more than 800 columns.

I have a SAS script :

filename fln "C:\file.txt";

DATA WORK.tableA;
....

RUN; filename fln;

But I get an error

ERROR: The submitted line exceeds maximum line length - 32768 bytes.

Help please.

Re: Loading Huge Dataset

Reeza — Fri, 27 Nov 2015 15:38:43 GMT

You may have missed a semicolon somewhere I think. Can you post your actual code?

Re: Loading Huge Dataset

Tom — Fri, 27 Nov 2015 15:45:20 GMT

Sounds like the source file for your program has lines that are way too long.

Perhaps the end of line characters got removed?

There is no need for any source code lines to be anywhere near 32K bytes long. In fact to make them readable by humans you should try to limit your source lines to 80 characters or less.

If you have lines of text that represent data then put them in a separate file rather than using inline data (CARDS or DATALINES statements). It appears that you might be trying to do that with the FILENAME statement that you posted.

Note that to read lines of data that is longer 32K you will need to use the LRECL option on the INFILE statement.

Re: Loading Huge Dataset

RW9 — Fri, 27 Nov 2015 15:54:47 GMT

I agree with Tom here, 800 columns is not a useable format. Return to the source and ask them to provide a useable dataset, you can use CSV, but break it up or normalize the data.

Re: Loading Huge Dataset

ilya_1991 — Mon, 30 Nov 2015 08:09:15 GMT

Thanks to everyone for help.
The reason was in code: it was generating automatically in the same line. When I added line breaks the problem has gone.