Severe penalty for using MFILE?

[yabwon]

It's already spinning, as soon as I get results will let you know. But for now it's taking much much longer then "plain data _null_".

Bart

[yabwon]

The macro I used was a bit different:

%macro genCode(genlineOfCode);
  %if %nrbquote(&genlineOfCode) = %nrbquote() %then %let genlineOfCode=100;
  %do i=1 %to &genlineOfCode;
    data _null_; 
      %do j = 1 %to &i.;
       a&j. = &j;
      %end;
    run;
  %end;
%mend;

but it looks like "the longer the generated data step code the bigger the difference", and it doesn't look "linear", I dare to say it's quadratic

Bart

Bart

[Tom]

That makes sense.  It is essentially opening the file again each time it wants to write.  And since it is writing at the end the larger the file the longer it takes to find the end to start writing.

[peterprec]

Looking at @yabwon 's code, I think it should be quadratic as you have 2 %do-loops with the same counter inside each other. Nice to see that theory and reality aligns. 🙂 And it still confirms the rather large penalty for using MFILE...

@Tom : Do you know something about the inner workings of the MFILE?