How do I split one text file into multiple text file according to file size limit

[andreas_lds]

Do you have to solve the problem using sas?

[paparock]

Yes. If doable, in SAS pls. TQ

[fja]

Does Kurt's reply solve your Problem?

[paparock]

Hi jfa,
Still trying but able to obtain the result. TQ

[Ksharp]

%let file= c:\temp\have.txt  ;     /*the original big text file*/
%let output=  c:\temp\   ;    /*the path you want to output*/
%let block=1  ;   /*Here is 1M for a file,if you need 30M for a file,specify 30*/


data _null_;
rc=filename('x',"&file.");
fid=fopen('x');
do i=1 to foptnum(fid);
  option=foptname(fid,i);
  value=finfo(fid,option);
  if i=4 then do;   
    size=input(value,best32.); /*the size of this file*/
    n= ceil(size/1024/1000/&block.);   
    call symputx('n',n);  /*put n into a macro variable*/
  end;
end;
run;

/*get the number of records of the splitted text file*/
data _null_;
 infile "&file." end=last;
 input;
 if last then do;
   length split $ 32000;
   f="%scan(&file.,-2,.\)";
   s="%scan(&file.,-1,.\)";
   size=ceil(_n_/&n.);
   do i=1 to &n.;
     if i=1 then split=catt("if _n_<",size," then fname='&output.\",f,"_",i,".",s,"';");
       else split=catt(split,"else if _n_<",i*size," then fname='&output.\",f,"_",i,".",s,"';");
   end;
   call symputx('split',split);
 end;
run;
%put &=n;
%put %bquote(&split) ;


/*start to split */
data _null_;
length fname $ 200;
&split.
infile "&file." lrecl=32767 ;
file dummy filevar=fname ;
input;
put _infile_;
run;

[Tom]

Is the file a TEXT file?  Consisting of LINES?  What is the maximum line length?

Assuming you are talking about a text file it is relatively easy to do in SAS.  It is easier to split by the number of RECORDS.

For example you might want to limit it to 10,000 lines per file.

filename in "original file name";
data _null_;
  infile in ;
  do line=1 to 10000;
      fileno+1;
      filename = cats("basename of file",fileno,".extension of file");
      file out filevar=filename;
      put _infile_;
  end;
run;

If you want to limit the size based on the number of BYTES written then it is a little harder.

For example to limit the files to 30,000,000 bytes you could do something like this:

%let size=30000000;

data _null_;
  infile in length=reclen end=eof;
  retain size &size fileno 0;
  length filename $256 ;
  input ;
  if (size + reclen +2 ) > &size then do;
    if _n_>1 then putlog _n_= fileno= size= ;
    fileno+1;
    size=0;
  end;
  filename = cats("c:\directory name\basename_",fileno,".txt");
  file out filevar=filename termstr=crlf ;
  put _infile_;
  size+reclen+2;
  if eof then putlog _n_= fileno= size= ;
run;