topic Re: Data step input in SAS Programming

Data step input

jacksonan123 — Wed, 02 Aug 2017 13:48:39 GMT

I have written the attached program to read a text file which it does okay.

data result  ;
retain flag_section1; length word $10; 
infile '/folders/myfolders/dlbootprocess/TEST1.smr/' TRUNCOVER ;
input @1 line $char800.; if line = ' ' then delete; if not flag_section then do;
  flag_section1 = index(line,'MINIMIZATION SUCCESSFUL');
  if flag_section1 then do;
    put flag_section1= _N_=;
    
  end;
end;
RUN;

The input file test1.smr is attached. I had to change the extension to *.txt in order to attach it to this question.

The output has a 1 for Minimization Successful and 0 values for all other lines of code. My question is how do I get it to retain the flagsection which has a 1 which is Minimization Successful and delete the remaining lines of code? The data is posted below and I have also attached the complete file.

THETA:      1)K13       2)K23       3)DURATION  4)LAG       5)LOGIT     6)K30       7)K34       8)VOLUME    9)K1315     10)K1415    11)DURATION 12)LAG      13)LOGIT    14)K150     15)K1516    16)VOLUME   17)WT       18)WT       
ETA:        19)         20)         21)         22)         23)         24)         25)         
ERR:        
TEST1.lst	-2681.582	FOCEI	eval=524 sig=3.2 sub=34 obs=1154 NM7.3.0 PP7.3.0 
THETA     = 3.22        2.18        0.9         10          -0.75       3.92        1.29        3000        0.273       0.0861      0.9         10          -0.75       43.9        1.29        3000        0.5         0.5

ETASD     = 0.671c      0.640c      0.947       0.126c      0.447c      0.000       0.305

ETAPval   = 0.199       0.036       0.189       0.000       0.000       0.207       0.008
ETAshr%   = 35.3        36.3        20.7        31.0        58.4        99.9        0.8
EBVshr%   = 16.3        10.9        11.0        30.6        19.4        99.9        2.1
EPSshr%   = 100.0       8.0         8.0

EPSSD     = 0.707c      0.500       0.018

MINIMIZATION SUCCESSFUL
HOWEVER, PROBLEMS OCCURRED WITH THE MINIMIZATION.
REGARD THE RESULTS OF THE ESTIMATION STEP CAREFULLY, AND ACCEPT THEM ONLY
AFTER CHECKING THAT THE COVARIANCE STEP PRODUCES REASONABLE OUTPUT.

 Ttot 587:27.61 Test 587:22.08 Tcov 0:0 Ttcl 0:5.53

Re: Data step input

andreas_lds — Wed, 02 Aug 2017 14:26:04 GMT

Sorry, i am not 100% sure that i understand your problem, maybe posting the expected output dataset can help.

Re: Data step input

jacksonan123 — Wed, 02 Aug 2017 14:29:16 GMT

The output data set would only contain the line Minimization Successful
which is line 12 of the current output data set.

Re: Data step input

Reeza — Wed, 02 Aug 2017 14:36:28 GMT

It looks like you're trying to caputre output from a log or output file. If this is from a SAS process there may be other ways, such as automatic macro variables or an ODS table that will allow you to avoid this. The alternative option, and several people have asked this recently, is to not import the file at all, but search it for a specific string and flag the file as string found or not. If you can pass X commands through to your OS, this is easily done via OS commands though it's still relatively straightforward with SAS.

If you clarify your issue we may be able to provide more 'optimal' code.

Re: Data step input

jacksonan123 — Wed, 02 Aug 2017 15:40:12 GMT

Yes I am actually only trying to capture the line which states Minimization
Successful or in some other cases Minimization terminated. From the file
that is the only line I would like to retain. I have not used OS commands
but they are explained on the SAS website.

Re: Data step input

ballardw — Wed, 02 Aug 2017 15:44:42 GMT

@jacksonan123 wrote:
The output data set would only contain the line Minimization Successful
which is line 12 of the current output data set.

And if the line does not occur in the file what is the output??

Re: Data step input

Tom — Wed, 02 Aug 2017 15:45:48 GMT

It is probably easiest to only write the record when you hit the end of the input file.

Make sure to initialize the flag to false.

You can also take advantage of the automatic _INFILE_ to simplify your code.

Your data file looks very structured so just test for the string at the beginning of the line. It really doesn't look to me like it should appear in the mddle of any lines.

data result  ;
  retain flag_section1 0;
  infile '/folders/myfolders/dlbootprocess/TEST1.smr' end=eof;
  input @;
  if _infile_ =: 'MINIMIZATION SUCCESSFUL' then flag_section1 =1;
  if eof then output;
run;

Re: Data step input

jacksonan123 — Wed, 02 Aug 2017 15:53:46 GMT

I ran your code but it only put out the number 1 with label flag_section 1
but not the line that goes with it ie "Minimization Successful.'

data result ;

retain flag_section1 0;

infile '/folders/myfolders/dlbootprocess/TEST1.smr' end=eof;

input @; if _infile_ =: 'MINIMIZATION SUCCESSFUL' then flag_section1 =1;

if eof then output;

run;

proc print data=result;

run;

Re: Data step input

jacksonan123 — Wed, 02 Aug 2017 15:55:46 GMT

The code returns the number 1 labelled as flag_section1 but not the line of
code Minimization Successful.

Re: Data step input

Tom — Wed, 02 Aug 2017 16:01:03 GMT

So you want to know what line in the source file had the string? Not sure why that adds any value.

You have all of the information and examples you need to fix that yourself.

Please do so and post the code you end up using.

Re: Data step input

jacksonan123 — Thu, 03 Aug 2017 12:41:59 GMT

The modified code allows for the distinction bewteen the two cases Minimization successful and Minimization Terminated whch can occur in the file.

data result ; 
/*inialization of flag to 0 which will be used to find line of interest*/
retain flag_section1 0 flag_section2 0; 
infile '/folders/myfolders/dlbootprocess/TEST4.smr' end=eof; 
input @; if _infile_ =: 'MINIMIZATION SUCCESSFUL' then flag_section1 =1; 
input @; if _infile_ =: 'MINIMIZATION TERMINATED' then flag_section2 =2; 
if eof then output;
 run;

proc print data=result;
run;