DATA Step, Macro, Functions and more

Exceeding the INPUT statement limits?

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 9
Accepted Solution

Exceeding the INPUT statement limits?

I'm trying to read in a comma delimited ascii file (CSV) that has variable names in row 1 using the INFILE and INPUT statements within a DATA STEP. The problem is that the length of row 1 exceeds the limit of the input statement (something like 32,767 characters). Does anyone have a strategy to work around that limit? Thanks!


Accepted Solutions
Solution
‎01-03-2013 12:52 PM
Respected Advisor
Posts: 3,777

Re: Exceeding the INPUT statement limits?

LRECL can be MUCH larger than 32767 depending on the OS 1gig or more I can't remember.  You won't be able to use _INFILE_ as a variable but then you shouldn't need to.

Show your work.

View solution in original post


All Replies
Solution
‎01-03-2013 12:52 PM
Respected Advisor
Posts: 3,777

Re: Exceeding the INPUT statement limits?

LRECL can be MUCH larger than 32767 depending on the OS 1gig or more I can't remember.  You won't be able to use _INFILE_ as a variable but then you shouldn't need to.

Show your work.

Occasional Contributor
Posts: 9

Re: Exceeding the INPUT statement limits?

Syntax

LRECL=n | nK | nM | nG | nT | hexX | MIN | MAX 

Syntax Description

n

specifies the logical record length in multiples of 1 (bytes); 1,024 (kilobytes); 1,048,576 (megabytes); 1,073,741,824 (gigabytes); or 1,099,511,627,776 (terabytes). For example, a value of 32 specifies 32 bytes, and a value of 32k specifies 32,767 bytes.

Default: 256

Range: 1-32767

hexX

specifies the logical record length as a hexadecimal value. You must specify the value beginning with a number (0-9), followed by an X. For example, the value 2dx sets the logical record length to 45 characters.

MIN

specifies a logical record length of 1.

MAX

specifies a logical record length of 32,767.

Respected Advisor
Posts: 3,777

Re: Exceeding the INPUT statement limits?

So what did you use?  Do you know the length of the longest record?

how about somthing like

filename FT44F001 "<path>" lrecl=1M;

You will need a value that is a least as long as the longest record.

Occasional Contributor
Posts: 9

Re: Exceeding the INPUT statement limits?

One example of a row 1 that is too long is one with a width equal to 35,265 characters.

Respected Advisor
Posts: 3,777

Re: Exceeding the INPUT statement limits?

Then set LRECL to that or larger.

Super User
Super User
Posts: 6,499

Re: Exceeding the INPUT statement limits?

SAS should have no trouble reading a line with more than 32K characters in it.

Can you be more specific about what you are actually running and the error message you are getting?

Super User
Posts: 5,080

Re: Exceeding the INPUT statement limits?

Do you really need to read the first row?  Why not just start with the second row:

infile source firstobs=2;

Occasional Contributor
Posts: 9

Re: Exceeding the INPUT statement limits?

I need the first row because it contains the variable names. The dataset would be of limited value if the variables were named col1-col4683. It would be possible to rename the variables by hand, but I would rather have an automated work around. I have at least 60 datasets to convert to SAS, with many more to come.

Respected Advisor
Posts: 3,777

Re: Exceeding the INPUT statement limits?

If you are going to use the names from the first row you will need a two step process.  1) to read the names 2) gen some code to use the names.  What about TYPE and INFORMAT.

Occasional Contributor
Posts: 9

Re: Exceeding the INPUT statement limits?

Even though my SAS installation SAS/BASE documentation says

     LRECL=MAX specifies a logical record length of 32,767.

and

     LRECL Range = 1 - 32,767

the online SAS for Windows companion states


     LRECL=record-length     specifies the record length (in bytes). Under Windows, the default is 256.     The value of record-length can range from 1 to 1,073,741,823 (1 gigabyte).

The moral of the story is:

- Don't trust the installed SAS/BASE documentation to be complete. I thought

  that they had hot links to specific host documentation where it was needed.

  Must be a hit-or-miss feature. Maybe that could be an item in the next SASware Ballot.

- Make sure to check the section entitled 'Using SAS Software in Your Operating Environment'

  or the online (regularly updated) 'SAS 9.3 Companion for Windows'

  SAS(R) 9.3 Companion for Windows

  before committing to a specific programming solution.

- Use the SAS Commuties forums like this one when you're stuck. Provide as much

  documentation as you can for questions and answers.

PROC Star
Posts: 7,360

Re: Exceeding the INPUT statement limits?

: Additional moral: pay close attention to what you are reading.  For some probably outdated reason the SAS system option for lrecl will be set to 32767 when one chooses MAX and, in fact, can NOT be set above that.

Of course, as you discovered, the infile lrecl option can be set to be much higher.

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 11 replies
  • 998 views
  • 3 likes
  • 5 in conversation