topic Re: series of do until to pick values out of a text file in SAS Programming

series of do until to pick values out of a text file

Caroline — Tue, 18 Dec 2018 13:03:12 GMT

I will be looping through a series of text files that are outputted from another program. I need to read through each file once, pulling out certain values to build a table. There are similarities to the files, but the line numbers will vary. I have been working with do until() to look for keywords, and then input the correct values. I can get each do until() group to work singly, but when I put them in series, it bombs out. I have attached a sample file.

data test; 
length test $34. model $80. ;
infile "path\PRI018\TKN.txt" obs=60  missover;
f="F";   g="F";  
   do until(f="T");
     input test $ 2-35 @;
     if test ne 'Number of Observations           :' then delete;
   else do;
     input @40 n //// @40 POR $9.;
     f="T";
     put _n_;		
   end;
   end;
put POR;
   do until(g='T');
     input test2 $ 2-11 @;
     if test2 ne 'Ln(Load) =' then delete;
   else do;
     input @13 model & $80.;
     g="T";
     put _n_;
   end;
   end;
put model;		
drop test test2  f g ;
run;

What am I missing?

I'm using SAS 9.4 1M2

Re: series of do until to pick values out of a text file

RW9 — Tue, 18 Dec 2018 13:17:19 GMT

You haven't actually said what you are trying to do or get out. If I want to find strings in a text file then:

data want;
  file...;
  input;
  if index(_infile_,"Number of Observations") or
     index(_infile_,"Ln(Load)") then output;
run;

You can then post process what is output from that.

Re: series of do until to pick values out of a text file

Caroline — Tue, 18 Dec 2018 13:28:49 GMT

I'm trying to get out a value that immediately follows what I've found. So if it finds "Ln(Load)=" then I want it to input the rest of that line "a0 + a1 LnQ + a2 LnQ^2 + a3 Sin(2 pi dtime) + a4 Cos(2 pi dtime)" as variable model. The values need to be input in order. There are some values that I need once, but appear several times (for instance, I may search for "AMLE", but it appears for streamflow and concentration, but I only want concentration).

There are roughly a dozen different variables that I need to pick out of each file.

Re: series of do until to pick values out of a text file

RW9 — Tue, 18 Dec 2018 13:39:15 GMT

What you have there is a bit of a mess, ideally going back to source and getting actual data would be easier. Anyways, if I had to do this, then I would just read in the whole file:

data want;
  length line $;
  file ...;
  input line $;
  retain flag1;
  if flag=1 then do;   /* I.e. row has been found, so observations now are data */
    ...;
  end;
  if flag=2 then do;
    ...;
  end;
  if index(...) then flag=1;
  if index(...) then flag=2;
...
run;

With that approach you can set numerous flags/counts etc. then just loop through the data, parsing what you need out.

Re: series of do until to pick values out of a text file

Astounding — Tue, 18 Dec 2018 13:41:00 GMT

When you put the loops together, here's what happens.

The top loop works.

The bottom loop begins, but the text "Ln(Load) = " doesn't appear for a few lines. The first time it is not found, the DELETE statement executes. This does not move you to the top of the DO UNTIL loop. Rather, it moves you to the top of the entire DATA step, so the top loop will execute again. And the top loop moves through the rest of the file without finding a match.

While other simplifications may be possible, the simplest would be to change the bottom DELETE statement to an assignment statement:

  if test2 ne 'Ln(Load) =' then g="F";

Re: series of do until to pick values out of a text file

Caroline — Tue, 18 Dec 2018 13:55:19 GMT

That explains the results I was getting. The assignment statement actually puts it into an infinite loop because of the trailing @ on the input test2 line. When I changed the line to

if test2 ne 'Ln(Load) =' then input;

it works perfectly.

thank you

Re: series of do until to pick values out of a text file

Astounding — Tue, 18 Dec 2018 14:07:54 GMT

Of course. Sorry I didn't catch that the first time. Good luck.