SAS Data Integration Studio, DataFlux Data Management Studio, SAS/ACCESS, SAS Data Loader for Hadoop and others

Loading Huge Dataset

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 9
Accepted Solution

Loading Huge Dataset

Good Day. 
I am trying to upload txt file with more than 800 columns.  

I have a SAS script : 

filename fln "C:\file.txt";

DATA WORK.tableA;
.... 

RUN; filename fln;

 

But I get an error 

ERROR: The submitted line exceeds maximum line length - 32768 bytes.

Help please. 


Accepted Solutions
Solution
‎11-30-2015 03:09 AM
Super User
Super User
Posts: 7,039

Re: Loading Huge Dataset

Posted in reply to ilya_1991

Sounds like the source file for your program has lines that are way too long.  

Perhaps the end of line characters got removed?

 

There is no need for any source code lines to be anywhere near 32K bytes long.  In fact to make them readable by humans you should try to limit your source lines to 80 characters or less.

 

If you have lines of text that represent data then put them in a separate file rather than using inline data (CARDS or DATALINES statements).  It appears that you might be trying to do that with the FILENAME statement that you posted.

 

Note that to read lines of data that is longer 32K you will need to use the LRECL option on the INFILE statement.

View solution in original post


All Replies
Super User
Posts: 19,770

Re: Loading Huge Dataset

Posted in reply to ilya_1991
You may have missed a semicolon somewhere I think. Can you post your actual code?
Solution
‎11-30-2015 03:09 AM
Super User
Super User
Posts: 7,039

Re: Loading Huge Dataset

Posted in reply to ilya_1991

Sounds like the source file for your program has lines that are way too long.  

Perhaps the end of line characters got removed?

 

There is no need for any source code lines to be anywhere near 32K bytes long.  In fact to make them readable by humans you should try to limit your source lines to 80 characters or less.

 

If you have lines of text that represent data then put them in a separate file rather than using inline data (CARDS or DATALINES statements).  It appears that you might be trying to do that with the FILENAME statement that you posted.

 

Note that to read lines of data that is longer 32K you will need to use the LRECL option on the INFILE statement.

Super User
Super User
Posts: 7,942

Re: Loading Huge Dataset

Posted in reply to ilya_1991

I agree with Tom here, 800 columns is not a useable format.  Return to the source and ask them to provide a useable dataset, you can use CSV, but break it up or normalize the data.  

Occasional Contributor
Posts: 9

Re: Loading Huge Dataset

Posted in reply to ilya_1991

Thanks to everyone for help.  
The reason was  in code: it was generating automatically in the same line. When I added line breaks the problem has gone. 

🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 4 replies
  • 670 views
  • 2 likes
  • 4 in conversation