How to determine a file's maximum record length

[#1]

To my great surprise or maybe incompetence, I couldn't find an article or usage note detailing how to find the maximum length of the records in a file. So I wrote a snippet and thought I could share it here. 

Here's one way to achieve this:

%* Create a file with 4 records;
data _null_;
  file "&wdir/t.txt" lrecl=%sysevalf(2**30-1);
  length A $32767;
  A=repeat('a',32766);
  put 'aa' / A A A / A / 'aaa';
run;

%* Find the record lengths in that file;
data _null_;
  infile "&wdir/t.txt" lrecl=%sysevalf(2**30-1) end=LASTREC length=LS;
  input ;      
  retain RECMINLEN RECMAXLEN RECNB;
  RECNB+1;
  RECMINLEN = min(RECMINLEN, LS);
  RECMAXLEN = max(RECMAXLEN, LS);
  if LASTREC then do;
    call symputx('recmaxlen', RECMAXLEN);
    call symputx('recminlen', RECMINLEN);
    call symputx('recnb    ', RECNB    );
  end;
run;

%* Print the record information;
%put &=recmaxlen &=recminlen &=recnb;

This prints out:

RECMAXLEN=98303 RECMINLEN=2 RECNB=4

Note that if the record length exceeds 1,073,741,824 (2^30) bytes, which is the maximum that SAS can handle, this method fails.

I'm looking forward to reading about ways to improve on the logic detailed.

[ChrisNZ]

Here is the logic encapsulated in a macro so it's easier to use.

/*****************************************************************************
  
  Name:               reclen.sas 

  Purpose:            This macro detects the length of a file's records and the record count 

  Jira:               None

  Type:               Macro Code

  Generates SAS code: No (Code pushed out to a call dosubl)

  Macros called:      None

******************************************************************************
 Who            When        What  
******************************************************************************
 C Graffeuille  14/02/2022  Initial version
*****************************************************************************/

%macro reclen(file, options);
/*  %macro _; %mend _;*/

  %************************************************;
  %**  I N I T                                   **;
  %************************************************;
  %local rc dsout do_log do_var do_var do_max do_min do_nb;

  %************************************************;
  %**  H E L P   S C R E E N                     **;
  %************************************************; 
  %if %qupcase(%superq(file))=HELP | ^%length(%superq(file)) %then %do;
    %let rc=%sysfunc(dosubl(%nrstr(
    %macro _; %mend _;
    options ps=max ls=132 nonotes nosymbolgen nomlogic msglevel=n;
    data _null_;
      LINE=repeat('#',99);
      putlog '00'x;
      putlog LINE;
      putlog '#                                                                                                  #';
      putlog '#           ____________________________                                             /\"_"/\       #';
      putlog "#           Help screen for macro RECLEN                                            ( ='.'= )      #";
      putlog '#           ----------------------------                                            (") "" (")     #';
      putlog '#                                                                                     |"  "|       #';
      putlog '#  This macro detects the length of a file''s records and the record count.           /"/  \"\      #';
      putlog '#    There are four ways to output the result.                                      (")"||"(")     #';
      putlog '#    The output can go to macro variables, to the log, to a data set                    ))         #';
      putlog '#      and/or be printed as a value                                                    ((          #';
      putlog '#  This macro generates no SAS code and can be used inside a data step. See below.      ))         #';
      putlog '#  Version 2023.02.16                                                                              #';
      putlog '#                                                                                                  #';
      putlog '#  Parameters                                                                                      #';
      putlog '#  ==========                                                                                      #';
      putlog '#                                                                                                  #';
      putlog '#    file name             REQD  File to analyse.                                                  #';
      putlog '#                                                                                                  #';
      putlog '#    options               OPTL  The macro can provide its results in up to 4 manners.             #';
      putlog '#                                The value of parameter options can be none, one, or many of:      #';
      putlog '#                                  LOG         To display values in the log window.                #';
      putlog '#                                  VAR         To store the values in global macro variables.      #';
      putlog '#                                               The variables are: reclenmax, reclenmin, recnb.    #';
      putlog '#                                  <data set>  To store the values in a data set.                  #';
      putlog '#                                  MIN MAX NB  To output the value as is. Only one of the three.   #';
      putlog '#                                Default: LOG VAR                                                  #';
      putlog '#                                                                                                  #';
      putlog '#  Examples                                                                                        #';
      putlog '#  ========                                                                                        #';
      putlog '#                                                                                                  #';
      putlog '#    %reclen( help )             Display this help screen                                          #';
      putlog '#                                                                                                  #';
      putlog '#    %reclen( c:\temp\t.csv )    Analyse the file and write the maximum & minimum record lengths   #';
      putlog '#                                  and record count to the log. Also save the values to macro      #';
      putlog '#                                  variables reclenmax, reclenmin, and recnb.                      #';
      putlog '#                                                                                                  #';
      putlog '#    %* create a flat file;                                                                        #';
      putlog '#    data _null_;                                                                                  #';
      putlog '#      file "&wdir/t.dat" lrecl=%sysevalf(2**30-1);                                                #';
      putlog '#      length A $32767;                                                                            #';
      putlog '#      A=repeat("a",32766);                                                                        #';
      putlog '#      put "aa" / A A A / A / "aaa";                                                               #';
      putlog '#    run;                                                                                          #';
      putlog '#                                                                                                  #';
      putlog '#    %reclen(&wdir/t.dat, LOG VAR WORK.RECLEN);     Save RECLEN results 1) in the log, 2) in       #';
      putlog '#                                                    macro variables and 3) in a data set.         #';
      putlog '#                                                                                                  #';
      putlog '#    %put &=reclenmax &=reclenmin &=recnb;          Print the value of the macro variables.        #';
      putlog '#                                                                                                  #';
      putlog '#    data _null_;                                   Use the result directly in your code.          #';
      putlog '#      put "NB=%reclen(&wdir/t.dat, NB)";                                                          #';
      putlog '#    run;                                                                                          #';
      putlog '#                                                                                                  #';
      putlog LINE;
      putlog '00'x;
    run;
    )));
    %return;
  %end;

  %************************************************;
  %**  M A I N                                   **;
  %************************************************; 
  %if %index(%superq(file),%str(%")) %then %do;
    %put ERROR: File name should not contain double quotes. Macro RECLEN will stop executing.; 
    %return;
  %end;

  %if ^%sysfunc(fileexist(%superq(file))) %then %do;
    %put ERROR: File %superq(file) not found. Macro RECLEN will stop executing.; 
    %return;
  %end;
  %let options=%upcase(%sysfunc(compress(%superq(options),%str( ).,kn))); 

  %if ^%length(&options.) %then %do;
    %let do_log=1;
    %let do_var=1;
  %end;
  %else %do;
    %if %sysfunc(findw(&options., LOG)) %then %let do_log =1;
    %if %sysfunc(findw(&options., VAR)) %then %let do_var =1;
    %if %sysfunc(findw(&options., MAX)) %then %let do_max =1;
    %if %sysfunc(findw(&options., MIN)) %then %let do_min =1;
    %if %sysfunc(findw(&options., NB )) %then %let do_nb  =1;
    %let options=%sysfunc(prxchange(s/\b(LOG|VAR|MAX|MIN|NB)\b//,-1,&options));
    %if %length(&options.)              %then %let dsout  =%scan(&options,1,%str( ));
  %end;

  %let rc = %sysfunc(dosubl(%nrstr(
    options ps=max ls=132 nonotes nosymbolgen nomlogic msglevel=n;
    data %sysfunc(coalescec(%str( &dsout.),_null_));    
      infile "&file" lrecl=%sysevalf(2**30-1) end=LASTREC length=RECLEN;
      input ;      
      retain RECLENMAX RECLENMIN RECNB;
      keep   RECLENMAX RECLENMIN RECNB;
      RECLENMAX = max(RECLENMAX, RECLEN);
      RECLENMIN = min(RECLENMIN, RECLEN);
      RECNB+1;
      if LASTREC then do;
        output;
        LINE=repeat('=',79);
        if 0&do_log.=1 then putlog LINE / "Record information for file &file" / LINE / RECLENMAX=  RECLENMIN= RECNB= / LINE;
        if 0&do_var.=1 then do;
          call symputx('reclenmax', RECLENMAX, 'g');
          call symputx('reclenmin', RECLENMIN, 'g');
          call symputx('recnb    ', RECNB    , 'g');
        end;
      end;
    run;
  )));           
  %if       &do_max.=1 %then &reclenmax; 
  %else %if &do_min.=1 %then &reclenmin; 
  %else %if &do_nb. =1 %then &recnb    ; 

  %************************************************;
  %**  T E R M                                   **;
  %************************************************; 

%mend;

[ChrisNZ]

To whom it may concern ( @ChrisHemedinger ? )

Can we have a larger code dialog? It's tiny.

[Tom]

The maximum value your SAS session can use for the LRECL= option on the INFILE statement depends on your computer.  My PC did not like to use a LRECL of 2**30-1 (1,073,741,823).  But it did not mind 1,000,000,000.

To generate a test file you can use @ cursor motion commands to make it simpler.

1243  options generic;
1244  filename test temp;
1245  data _null_;
1246    file test lrecl=%sysevalf(2**30-1);
1247    put 'aa' / @%sysevalf(2**30-1) 'a' / / 'aaa';
1248  run;

NOTE: The file TEST is:
      (system-specific pathname),
      (system-specific file attributes)

NOTE: 4 records were written to the file (system-specific pathname).
      The minimum record length was 0.
      The maximum record length was 1073741823.
NOTE: DATA statement used (Total process time):
      real time           1.42 seconds
      cpu time            0.78 seconds

[ChrisNZ]

Good idea to use the @ pointer @Tom, thank  you. 

Your system does seems to accept a value of lrecl = 2**30-1 in your code above? 

[Tom]

Yes.  Input seems to need more memory than output.

[ChrisNZ]

Weird that I can use the max length for both (with MEMSIZE=16GB, version M7, if that matters).
I'm really curious why you can't. I'd ask Tech support.