DATA Step, Macro, Functions and more

Data step input

Accepted Solution Solved
Reply
Frequent Contributor
Posts: 83
Accepted Solution

Data step input

I have written the attached program to read a text file which it does okay.

data result  ;
retain flag_section1; length word $10; 
infile '/folders/myfolders/dlbootprocess/TEST1.smr/' TRUNCOVER ;
input @1 line $char800.; if line = ' ' then delete; if not flag_section then do;
  flag_section1 = index(line,'MINIMIZATION SUCCESSFUL');
  if flag_section1 then do;
    put flag_section1= _N_=;
    
  end;
end;
RUN;

The input file test1.smr is attached.  I had to change the extension to *.txt in order to attach it to this question.

The output has a 1 for Minimization Successful and 0 values for all other lines of code.  My question is how do I get it to retain the flagsection which has a 1 which is Minimization Successful and delete the remaining lines of code?  The data is posted below and I have also attached the complete file.

THETA:      1)K13       2)K23       3)DURATION  4)LAG       5)LOGIT     6)K30       7)K34       8)VOLUME    9)K1315     10)K1415    11)DURATION 12)LAG      13)LOGIT    14)K150     15)K1516    16)VOLUME   17)WT       18)WT       
ETA:        19)         20)         21)         22)         23)         24)         25)         
ERR:        
TEST1.lst	-2681.582	FOCEI	eval=524 sig=3.2 sub=34 obs=1154 NM7.3.0 PP7.3.0 
THETA     = 3.22        2.18        0.9         10          -0.75       3.92        1.29        3000        0.273       0.0861      0.9         10          -0.75       43.9        1.29        3000        0.5         0.5

ETASD     = 0.671c      0.640c      0.947       0.126c      0.447c      0.000       0.305

ETAPval   = 0.199       0.036       0.189       0.000       0.000       0.207       0.008
ETAshr%   = 35.3        36.3        20.7        31.0        58.4        99.9        0.8
EBVshr%   = 16.3        10.9        11.0        30.6        19.4        99.9        2.1
EPSshr%   = 100.0       8.0         8.0

EPSSD     = 0.707c      0.500       0.018

MINIMIZATION SUCCESSFUL
HOWEVER, PROBLEMS OCCURRED WITH THE MINIMIZATION.
REGARD THE RESULTS OF THE ESTIMATION STEP CAREFULLY, AND ACCEPT THEM ONLY
AFTER CHECKING THAT THE COVARIANCE STEP PRODUCES REASONABLE OUTPUT.

 Ttot 587:27.61 Test 587:22.08 Tcov 0:0 Ttcl 0:5.53

Accepted Solutions
Solution
2 weeks ago
Super User
Super User
Posts: 6,500

Re: Data step input

So you want to know what line in the source file had the string?  Not sure why that adds any value.

You have all of the information and examples you need to fix that yourself.

Please do so and post the code you end up using.

 

View solution in original post


All Replies
Super Contributor
Posts: 259

Re: Data step input

Sorry, i am not 100% sure that i understand your problem, maybe posting the expected output dataset can help.

Frequent Contributor
Posts: 83

Re: Data step input

The output data set would only contain the line Minimization Successful
which is line 12 of the current output data set.
Super User
Posts: 10,500

Re: Data step input


jacksonan123 wrote:
The output data set would only contain the line Minimization Successful
which is line 12 of the current output data set.

And if the line does not occur in the file what is the output??

Frequent Contributor
Posts: 83

Re: Data step input

I ran your code but it only put out the number 1 with label flag_section 1
but not the line that goes with it ie "Minimization Successful.'

data result ;

retain flag_section1 0;

infile '/folders/myfolders/dlbootprocess/TEST1.smr' end=eof;

input @; if _infile_ =: 'MINIMIZATION SUCCESSFUL' then flag_section1 =1;

if eof then output;

run;



proc print data=result;

run;


Super User
Posts: 17,828

Re: Data step input

It looks like you're trying to caputre output from a log or output file. If this is from a SAS process there may be other ways, such as automatic macro variables or an ODS table that will allow you to avoid this. The alternative option, and several people have asked this recently, is to not import the file at all, but search it for a specific string and flag the file as string found or not. If you can pass X commands through to your OS, this is easily done via OS commands though it's still relatively straightforward with SAS. 

 

If you clarify your issue we may be able to provide more 'optimal' code.

Frequent Contributor
Posts: 83

Re: Data step input

Yes I am actually only trying to capture the line which states Minimization
Successful or in some other cases Minimization terminated. From the file
that is the only line I would like to retain. I have not used OS commands
but they are explained on the SAS website.
Super User
Super User
Posts: 6,500

Re: Data step input

It is probably easiest to only write the record when you hit the end of the input file.

Make sure to initialize the flag to false.

You can also take advantage of the automatic _INFILE_ to simplify your code.

Your data file looks very structured so just test for the string at the beginning of the line.  It really doesn't look to me like it should appear in the mddle of any lines.

data result  ;
  retain flag_section1 0;
  infile '/folders/myfolders/dlbootprocess/TEST1.smr' end=eof;
  input @;
  if _infile_ =: 'MINIMIZATION SUCCESSFUL' then flag_section1 =1;
  if eof then output;
run;

 

Frequent Contributor
Posts: 83

Re: Data step input

The code returns the number 1 labelled as flag_section1 but not the line of
code Minimization Successful.
Solution
2 weeks ago
Super User
Super User
Posts: 6,500

Re: Data step input

So you want to know what line in the source file had the string?  Not sure why that adds any value.

You have all of the information and examples you need to fix that yourself.

Please do so and post the code you end up using.

 

Frequent Contributor
Posts: 83

Re: Data step input

The modified code allows for the distinction bewteen the two cases Minimization successful and Minimization Terminated whch can occur in the file.

data result ; 
/*inialization of flag to 0 which will be used to find line of interest*/
retain flag_section1 0 flag_section2 0; 
infile '/folders/myfolders/dlbootprocess/TEST4.smr' end=eof; 
input @; if _infile_ =: 'MINIMIZATION SUCCESSFUL' then flag_section1 =1; 
input @; if _infile_ =: 'MINIMIZATION TERMINATED' then flag_section2 =2; 
if eof then output;
 run;

proc print data=result;
run;
☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 10 replies
  • 199 views
  • 0 likes
  • 5 in conversation