DATA Step, Macro, Functions and more

Input string with an unknown variable length

Accepted Solution Solved
Reply
Contributor
Posts: 28
Accepted Solution

Input string with an unknown variable length

Hi, Hopefully a simple question.

I need to input a string variable from a csv file that can be of any length from 0 to 32K characters. The maximum length is unkown at the time of import and I would like to set the variable size to the actual maximum length in the csv file. So I think I need to use $varyingw but looking at examples it seems like the length needs to be defined in the input file or is set to the record length. What is the best way of inputting this field?

Thanks for your help.


Accepted Solutions
Solution
‎01-11-2014 02:02 PM
Occasional Contributor
Posts: 14

Re: Input string with an unknown variable length

In such cases I sometimes read file with maximum length (32767, e.g.) and then decrease the length of the variable down to the maximum length of values.

proc sql noprint;

     select max(length(variable)) into :maxlen

     from mydata;

quit;

options varlenchk=nowarn;

data mydata;

     length variable $&maxlen;

     set mydata;

run;

options varlenchk=warn;

VARLENCHK-option is necessary if you want to avoid warning from SAS that you may truncate some values.

View solution in original post


All Replies
Super User
Super User
Posts: 7,077

Re: Input string with an unknown variable length

If you are reading it from a CSV file you do not need to do anything other than define the variable long enough.

data want ;

  infile 'myfile.csv' dsd lrecl=2000000 ;

  length id 8 string $32767 ;

  input id string;

run;

Contributor
Posts: 28

Re: Input string with an unknown variable length

Thanks Tom. So is there any disc space issues in setting the length so big? Also, if the maximum length was 259 characters (for example, but I dont know this beforehand) is there is no way I can implement the input statement to default to 259 when executed?

Super User
Super User
Posts: 7,077

Re: Input string with an unknown variable length

If you know that the maximum possible length then define the variable using that length.  So in my example change the 32767 (which is maximum possible variable length) to 259.  You can also change the LRECL on the INFILE statement, but I doubt that would have much impact on performance.

SAS stores all character variables as fixed length.  But if you use the COMPRESS dataset option then the extra empty space does not take much space on the disk.

Super Contributor
Super Contributor
Posts: 444

Re: Input string with an unknown variable length

is 2000000 the max lrecl,Tom?

Super User
Super User
Posts: 7,077

Re: Input string with an unknown variable length

I am sure it depends on the operating system and SAS version. On SAS 9.2 running on 64 bit unix I can use LRECL up to

1,073,741,823 (which 2**31 -1) without getting a syntax error.

Super Contributor
Super Contributor
Posts: 444

Re: Input string with an unknown variable length

thanks Tom

Solution
‎01-11-2014 02:02 PM
Occasional Contributor
Posts: 14

Re: Input string with an unknown variable length

In such cases I sometimes read file with maximum length (32767, e.g.) and then decrease the length of the variable down to the maximum length of values.

proc sql noprint;

     select max(length(variable)) into :maxlen

     from mydata;

quit;

options varlenchk=nowarn;

data mydata;

     length variable $&maxlen;

     set mydata;

run;

options varlenchk=warn;

VARLENCHK-option is necessary if you want to avoid warning from SAS that you may truncate some values.

🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 7 replies
  • 1539 views
  • 4 likes
  • 4 in conversation