topic Re: Exceeding the INPUT statement limits? in SAS Programming

Exceeding the INPUT statement limits?

gkeeler — Thu, 03 Jan 2013 16:48:32 GMT

I'm trying to read in a comma delimited ascii file (CSV) that has variable names in row 1 using the INFILE and INPUT statements within a DATA STEP. The problem is that the length of row 1 exceeds the limit of the input statement (something like 32,767 characters). Does anyone have a strategy to work around that limit? Thanks!

Re: Exceeding the INPUT statement limits?

data_null__ — Thu, 03 Jan 2013 17:52:02 GMT

LRECL can be MUCH larger than 32767 depending on the OS 1gig or more I can't remember. You won't be able to use _INFILE_ as a variable but then you shouldn't need to.

Show your work.

Re: Exceeding the INPUT statement limits?

Astounding — Thu, 03 Jan 2013 18:21:54 GMT

Do you really need to read the first row? Why not just start with the second row:

infile source firstobs=2;

Re: Exceeding the INPUT statement limits?

gkeeler — Thu, 03 Jan 2013 18:41:33 GMT

Syntax

LRECL=n | nK | nM | nG | nT | hexX | MIN | MAX

Syntax Description

specifies the logical record length in multiples of 1 (bytes); 1,024 (kilobytes); 1,048,576 (megabytes); 1,073,741,824 (gigabytes); or 1,099,511,627,776 (terabytes). For example, a value of 32 specifies 32 bytes, and a value of 32k specifies 32,767 bytes.

Default: 256

Range: 1-32767

hexX

specifies the logical record length as a hexadecimal value. You must specify the value beginning with a number (0-9), followed by an X. For example, the value 2dx sets the logical record length to 45 characters.

MIN

specifies a logical record length of 1.

MAX

specifies a logical record length of 32,767.

Re: Exceeding the INPUT statement limits?

gkeeler — Thu, 03 Jan 2013 18:49:35 GMT

I need the first row because it contains the variable names. The dataset would be of limited value if the variables were named col1-col4683. It would be possible to rename the variables by hand, but I would rather have an automated work around. I have at least 60 datasets to convert to SAS, with many more to come.

Re: Exceeding the INPUT statement limits?

data_null__ — Thu, 03 Jan 2013 19:11:14 GMT

So what did you use? Do you know the length of the longest record?

how about somthing like

filename FT44F001 "<path>" lrecl=1M;

You will need a value that is a least as long as the longest record.

Re: Exceeding the INPUT statement limits?

data_null__ — Thu, 03 Jan 2013 19:13:17 GMT

If you are going to use the names from the first row you will need a two step process. 1) to read the names 2) gen some code to use the names. What about TYPE and INFORMAT.

Re: Exceeding the INPUT statement limits?

gkeeler — Thu, 03 Jan 2013 19:21:22 GMT

One example of a row 1 that is too long is one with a width equal to 35,265 characters.

Re: Exceeding the INPUT statement limits?

data_null__ — Thu, 03 Jan 2013 19:28:42 GMT

Then set LRECL to that or larger.

Re: Exceeding the INPUT statement limits?

Tom — Thu, 03 Jan 2013 19:31:07 GMT

SAS should have no trouble reading a line with more than 32K characters in it.

Can you be more specific about what you are actually running and the error message you are getting?

Re: Exceeding the INPUT statement limits?

gkeeler — Tue, 08 Jan 2013 19:57:33 GMT

Even though my SAS installation SAS/BASE documentation says

LRECL=MAX specifies a logical record length of 32,767.

and

LRECL Range = 1 - 32,767

the online SAS for Windows companion states

LRECL=record-length specifies the record length (in bytes). Under Windows, the default is 256. The value of record-length can range from 1 to 1,073,741,823 (1 gigabyte).

The moral of the story is:

- Don't trust the installed SAS/BASE documentation to be complete. I thought

that they had hot links to specific host documentation where it was needed.

Must be a hit-or-miss feature. Maybe that could be an item in the next SASware Ballot.

- Make sure to check the section entitled 'Using SAS Software in Your Operating Environment'

or the online (regularly updated) 'SAS 9.3 Companion for Windows'

SAS(R) 9.3 Companion for Windows

before committing to a specific programming solution.

- Use the SAS Commuties forums like this one when you're stuck. Provide as much

documentation as you can for questions and answers.

Re: Exceeding the INPUT statement limits?

art297 — Tue, 08 Jan 2013 20:24:37 GMT

: Additional moral: pay close attention to what you are reading. For some probably outdated reason the SAS system option for lrecl will be set to 32767 when one chooses MAX and, in fact, can NOT be set above that.

Of course, as you discovered, the infile lrecl option can be set to be much higher.