@@, missover and flowover - difficulties when reading simple raw data

[DmytroYermak]

Hi all,

Could you please help to resolve the following task. We have raw data (datalines) of days and temperature that can be read by program 1:

/*program1*/

data test;
	input day temperature @@;
datalines;
5 21 6 23 7 22 1 23 2 23
3 24 4 25 6 26 7 23
;
run;

The program1 is working perfect.

Then we 'miss' the last value in line 1 - "23". The program2 is working incorrectly as flowover reads next value "3" in line 2.

Could you please help to mend the program. The datalines should be left as is.

/*program2*/

data test;
	input day temperature @@;
datalines;
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;

Thank you!

[Shmuel]

Add a line:

data test;
 infile datalines truncover;     /* line added */
 input day temperature @@;
datalines;
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;

[DmytroYermak]

Thank you. I tried on SAS Base 9.4 and SAS Studio and got my laptop 'hanged up'. It seems that trunkover is not functioning.

[Shmuel]

Next code was tested and works fine:

data test;
 infile datalines pad eov=done;   
 if not done then 
    input day temperature @@;
datalines;
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;

[DmytroYermak]

Hi Shmuel,

Unfortunately your code is not working as desirable:

[Astounding]

Here's a variation that ought to work (but untested):

data test;
  infile cards missover;
  input day temperature @@;
  if day > . then output;
  if temperature = . then input;
datalines;
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;

[DmytroYermak]

Hi Astounding,

Unfortunatelly I receve an error too:

155  data test;  infile cards missover;
156    input day temperature @@;  if day > . then output;  if temperature = . then input;
157  cards;

ERROR: The INFILE statement MISSOVER option and the INPUT statement double trailing @ option, are
       being used in an inconsistent manner.  The execution of the DATA STEP is being terminated to
       prevent an infinite loop condition.
RULE:      ----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9-
158        5 21 6 23 7 22 1 23 2
day=5 temperature=21 _ERROR_=1 _N_=1
NOTE: The SAS System stopped processing this step because of errors.

[Ksharp · Accepted solution]

data test;
infile cards truncover;
input day temperature @;
do while(not missing(day));
 output;
 input day temperature @;
end;
datalines;
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;

[DmytroYermak]

Thank you! I would never find it on my own.

[DmytroYermak]

Thanks to offered solutions I have found another one. Could you please criticize.

data test;
	infile datalines truncover;
	input code $1. price @@;
	if not missing(code) then output;
	  else input code $1. price;
datalines; 
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
run;

[Tom]

---

@DmytroYermak wrote:

Thanks to offered solutions I have found another one. Could you please criticize.

data test;
	infile datalines truncover;
	input code $1. price @@;
	if not missing(code) then output;
	  else input code $1. price;
datalines; 
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
run;

---

Looks good.

@You do not need to specify any variables in the second INPUT statement since it is just there to clear the current input line to prevent the combination of @@ and TRUNCOVER of causing an infinite loop.

Also use only a  semi-colon (or four of them if using CARDS4 instead of CARDS) instead of a some statement like RUN; to terminate the in-line data. Any characters before the semi-colon will be ignored so having them there is probably confusing for the humans reading the code.

Note that the real problem here is the data stream. If you want to use this type of list mode data stream then make sure to put in periods to represent missing data points.

[DmytroYermak]

Thank you for the detailed analysis. Here it is the updated code:

data test;
	infile datalines truncover;
	input code $1. price @@;
	if not missing(code) then output;
	  else input;
datalines; 
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;

I have just one question. Why on the second iteration (after "else input") new operator - "input code $1. price @@ " does not 'jump'(not reading the second line) on the third line. For instance in the following code:

data test;
	infile datalines truncover;
	input code $1. price @@;
	if not missing(code) then output;
	  else input;
datalines; 
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
5 21 6 23 7 22 1 23 2
3 24 4 25 6 26 7 23
;
run;
proc print;run;

 Does it mean that "@@" prescribes 'Stay on the same line which was read before' ?

[Tom]

---

DmytroYermak wrote:

---

I have just one question. Why on the second iteration (after "else input") new operator - "input code $1. price @@ " does not 'jump'(not reading the second line) on the third line. Does it mean that "@@" prescribes 'Stay on the same line which was read before' ?

Normally an INPUT statement 'finishes' the line of data so that the next input will read from the next line, whether that next INPUT is in the current iteration of the data step or the next.  That is why the INPUT in the ELSE forces SAS to move onto the next line.

A single @ will prevent the INPUT statement from 'finishing' with the line of data, but it will still be discarded at the end of the iteration of the data step.  You can use this if you only want to keep reading from the same line in the CURRENT iteration of the data step.  I think one example in this thread did that using a DO loop around the INPUT statement to execute the INPUT multiple times in the same iteration of the data step.

The @@ adds the additional feature that it prevents SAS from discarding the line when it starts a new iteration of the data step.  The most common use is for data like this where the data for multiple output observations are contained in a single line of input.

Combining @@ with TRUNCOVER can lead to infinite loops (also demonstrated in this thread) because the TRUNCOVER prevents SAS from going to a new line in search of input fields and the @@ prevents SAS from going to a new line when it makes another iteration of the data step and so there is no way for it to ever advance past the first line. (This also applies if you use the deprecated MISSOVER option instead of the TRUNCOVER option.)

[DmytroYermak]

---

@Tom wrote:

the TRUNCOVER prevents SAS from going to a new line in search of input fields

---

Do I understand correctly that SAS does not move on the third line (after reading first values in second line) just due to TRUNCOVER? In spite of new iteration?

[Tom]

---

@DmytroYermak wrote:

---

@Tom wrote:

the TRUNCOVER prevents SAS from going to a new line in search of input fields

---

Do I understand correctly that SAS does not move on the third line (after reading first values in second line) just due to TRUNCOVER? In spite of new iteration?

---

The TRUNCOVER option prevents it from moving to a new line when there are not enough values left on the line to satisfy the list of vairable on the INPUT statement, whether that was the first iteration that read from that line or the 10,000th.