DATA Step, Macro, Functions and more

@@, missover and flowover - difficulties when reading simple raw data

Accepted Solution Solved
Reply
Frequent Contributor
Posts: 142
Accepted Solution

@@, missover and flowover - difficulties when reading simple raw data

[ Edited ]

Hi all,

Could you please help to resolve the following task. We have raw data (datalines) of days and temperature that can be read by program 1:

/*program1*/

data test; input day temperature @@; datalines; 5 21 6 23 7 22 1 23 2 23 3 24 4 25 6 26 7 23 ; run;

The program1 is working perfect.

Then we 'miss' the last value in line 1 - "23". The program2 is working incorrectly as flowover reads next value "3" in line 2.

Could you please help to mend the program. The datalines should be left as is.

/*program2*/

data test;
	input day temperature @@;
datalines;
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;

Thank you!


Accepted Solutions
Solution
‎07-26-2017 10:00 AM
Super User
Posts: 10,035

Re: @@, missover and flowover - difficulties when reading simple raw data

Posted in reply to DmytroYermak
data test;
infile cards truncover;
input day temperature @;
do while(not missing(day));
 output;
 input day temperature @;
end;
datalines;
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;

View solution in original post


All Replies
Trusted Advisor
Posts: 1,570

Re: @@, missover and flowover - difficulties when reading simple raw data

Posted in reply to DmytroYermak

Add a line:

data test;
 infile datalines truncover;     /* line added */
 input day temperature @@;
datalines;
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;
Frequent Contributor
Posts: 142

Re: @@, missover and flowover - difficulties when reading simple raw data

[ Edited ]

Thank you. I tried on SAS Base 9.4 and SAS Studio and got my laptop 'hanged up'. It seems that trunkover is not functioning.printscreen.jpg

Trusted Advisor
Posts: 1,570

Re: @@, missover and flowover - difficulties when reading simple raw data

Posted in reply to DmytroYermak

Next code was tested and works fine:

data test;
 infile datalines pad eov=done;   
 if not done then 
    input day temperature @@;
datalines;
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;
Frequent Contributor
Posts: 142

Re: @@, missover and flowover - difficulties when reading simple raw data

Hi Shmuel,

 

Unfortunately your code is not working as desirable:

printscreen2.jpg

printscreen3.jpg

Super User
Posts: 5,511

Re: @@, missover and flowover - difficulties when reading simple raw data

Posted in reply to DmytroYermak

Here's a variation that ought to work (but untested):

 

data test;
infile cards missover; input day temperature @@;
if day > . then output;
if temperature = . then input; datalines; 5 21 6 23 7 22 1 23 2 3 24 4 25 6 26 7 23 ;

 

Frequent Contributor
Posts: 142

Re: @@, missover and flowover - difficulties when reading simple raw data

Posted in reply to Astounding

Hi Astounding,

 

Unfortunatelly I receve an error too:

155  data test;  infile cards missover;
156    input day temperature @@;  if day > . then output;  if temperature = . then input;
157  cards;

ERROR: The INFILE statement MISSOVER option and the INPUT statement double trailing @ option, are
       being used in an inconsistent manner.  The execution of the DATA STEP is being terminated to
       prevent an infinite loop condition.
RULE:      ----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9-
158        5 21 6 23 7 22 1 23 2
day=5 temperature=21 _ERROR_=1 _N_=1
NOTE: The SAS System stopped processing this step because of errors.
Solution
‎07-26-2017 10:00 AM
Super User
Posts: 10,035

Re: @@, missover and flowover - difficulties when reading simple raw data

Posted in reply to DmytroYermak
data test;
infile cards truncover;
input day temperature @;
do while(not missing(day));
 output;
 input day temperature @;
end;
datalines;
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;
Frequent Contributor
Posts: 142

Re: @@, missover and flowover - difficulties when reading simple raw data

Thank you! I would never find it on my own.Smiley Surprised

Frequent Contributor
Posts: 142

Re: @@, missover and flowover - difficulties when reading simple raw data

[ Edited ]

Thanks to offered solutions I have found another one. Could you please criticize.

data test;
	infile datalines truncover;
	input code $1. price @@;
	if not missing(code) then output;
	  else input code $1. price;
datalines; 
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
run;

 

Super User
Super User
Posts: 7,059

Re: @@, missover and flowover - difficulties when reading simple raw data

Posted in reply to DmytroYermak

DmytroYermak wrote:

Thanks to offered solutions I have found another one. Could you please criticize.

data test;
	infile datalines truncover;
	input code $1. price @@;
	if not missing(code) then output;
	  else input code $1. price;
datalines; 
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
run;

 


Looks good.

 

You do not need to specify any variables in the second INPUT statement since it is just there to clear the current input line to prevent the combination of @@ and TRUNCOVER of causing an infinite loop.

 

Also use only a  semi-colon (or four of them if using CARDS4 instead of CARDS) instead of a some statement like RUN; to terminate the in-line data. Any characters before the semi-colon will be ignored so having them there is probably confusing for the humans reading the code.

 

Note that the real problem here is the data stream. If you want to use this type of list mode data stream then make sure to put in periods to represent missing data points.

 

Frequent Contributor
Posts: 142

Re: @@, missover and flowover - difficulties when reading simple raw data

[ Edited ]

Thank you for the detailed analysis. Here it is the updated code:

 

data test;
	infile datalines truncover;
	input code $1. price @@;
	if not missing(code) then output;
	  else input;
datalines; 
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;

I have just one question. Why on the second iteration (after "else input") new operator - "input code $1. price @@ " does not 'jump'(not reading the second line) on the third line. For instance in the following code:

data test;
	infile datalines truncover;
	input code $1. price @@;
	if not missing(code) then output;
	  else input;
datalines; 
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;
proc print;run;

 Does it mean that "@@" prescribes 'Stay on the same line which was read before' ?

 

Super User
Super User
Posts: 7,059

Re: @@, missover and flowover - difficulties when reading simple raw data

Posted in reply to DmytroYermak

DmytroYermak wrote:

 


I have just one question. Why on the second iteration (after "else input") new operator - "input code $1. price @@ " does not 'jump'(not reading the second line) on the third line. Does it mean that "@@" prescribes 'Stay on the same line which was read before' ?

Normally an INPUT statement 'finishes' the line of data so that the next input will read from the next line, whether that next INPUT is in the current iteration of the data step or the next.  That is why the INPUT in the ELSE forces SAS to move onto the next line.

 

A single @ will prevent the INPUT statement from 'finishing' with the line of data, but it will still be discarded at the end of the iteration of the data step.  You can use this if you only want to keep reading from the same line in the CURRENT iteration of the data step.  I think one example in this thread did that using a DO loop around the INPUT statement to execute the INPUT multiple times in the same iteration of the data step.

 

The @@ adds the additional feature that it prevents SAS from discarding the line when it starts a new iteration of the data step.  The most common use is for data like this where the data for multiple output observations are contained in a single line of input.

 

Combining @@ with TRUNCOVER can lead to infinite loops (also demonstrated in this thread) because the TRUNCOVER prevents SAS from going to a new line in search of input fields and the @@ prevents SAS from going to a new line when it makes another iteration of the data step and so there is no way for it to ever advance past the first line. (This also applies if you use the deprecated MISSOVER option instead of the TRUNCOVER option.)

 

Frequent Contributor
Posts: 142

Re: @@, missover and flowover - difficulties when reading simple raw data


Tom wrote:
the TRUNCOVER prevents SAS from going to a new line in search of input fields

 


Do I understand correctly that SAS does not move on the third line (after reading first values in second line) just due to TRUNCOVER? In spite of new iteration?

Super User
Super User
Posts: 7,059

Re: @@, missover and flowover - difficulties when reading simple raw data

Posted in reply to DmytroYermak

DmytroYermak wrote:

Tom wrote:
the TRUNCOVER prevents SAS from going to a new line in search of input fields

 


Do I understand correctly that SAS does not move on the third line (after reading first values in second line) just due to TRUNCOVER? In spite of new iteration?


The TRUNCOVER option prevents it from moving to a new line when there are not enough values left on the line to satisfy the list of vairable on the INPUT statement, whether that was the first iteration that read from that line or the 10,000th.

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 14 replies
  • 393 views
  • 3 likes
  • 5 in conversation