Can someone explain the answer code for this challenge question?

[sfletc30]

data snowforecast;
    set winter2015_2016;
    retain FirstSnow;
    by Code;
    if first.Code then FirstSnow=Date;
    if last.Code then do;
        LastSnow=Date;
        WinterLengthWeeks=intck('week', FirstSnow, LastSnow, 'c');
        ProjectedFirstSnow=intnx('year', FirstSnow, 1, 'same');
        output;
    end;
    format FirstSnow LastSnow ProjectedFirstSnow date7.;
    drop Snow Date;	
run;

Hey there. I'm currently learning the SAS Programming 2 Module and I got to this challenge question. I'm using SAS OnDemand to answer the questions. I'm having trouble understanding why the code above is the correct answer. 

This is my original code:

data snowforecast;
	set winter2015_2016;
	by Code;
	if first.Code=1 then FirstSnow=Date;
	if last.Code=1 then LastSnow=Date;
	WinterLengthWeeks=intck('Week', FirstSnow, LastSnow, 'C');
	ProjectedFirstSnow=intnx('Year', FirstSnow, 1, 'same');
	format FirstSnow LastSnow ProjectedFirstSnow mmddyy.;
	drop Snow Date;
run;

My original code resulted in the table below (the missing values went on for almost all the rows to the end):

What I'm having trouble understanding are three things:

1. What does the retain statement do here for this code?

2. Why is an If-Then-Do statement necessary here? Why do those assignment statements have to go under the last.Code=1 part?

3. I introduced all the missing parts from the answer code into my code one by one. As soon as I included "output;", the table went from about 140+ rows to only 4. I don't understand how output did this here.

I have an introductory understanding of these concepts but this was my first time struggling to grasp them in a challenge question. Please help, thanks!

[LinusH]

1. Since there's an explicit OUTPUT when last.code, you need to retain the value of firstsnow across observations. All variables are reset for each observation, RETAIN prevents that to happen for selected variable.

2. This set of statements makes sense only to execute when you are at the last observation for your by group /code).

3. In a regular data step, there is an implicit OUTPUT at the end - usually at RUN. When introducing an explicit OUTPUT, there is no implicit OUTPUT any more, and you only write  an observation at the OUTPUT statement. In this case when the condition last.code is true.

[sfletc30]

Hey so just a followup question for number 3. So I understand what the code is saying now. If last.Code=1, then do these things (which includes writing to the output table with the OUTPUT statement). And last.Code=1 only exists four times in the dataset. So that explains why the rows went from around 140 to only 4.

But why do I have to explicitly tell SAS this? Why doesn't the implicit OUTPUT statement at the end of the code create the four rows that the explicit one does?

[Kurt_Bremser]

---

@sfletc30 wrote:

Hey so just a followup question for number 3. So I understand what the code is saying now. If last.Code=1, then do these things (which includes writing to the output table with the OUTPUT statement). And last.Code=1 only exists four times in the dataset. So that explains why the rows went from around 140 to only 4.

But why do I have to explicitly tell SAS this? Why doesn't the implicit OUTPUT statement at the end of the code create the four rows that the explicit one does?

---

Because the implicit OUTPUT writes an observation for every iteration of the DATA step, unless the iteration is cut short by a subsetting IF. And the number of iterations is (in your case) determined by the observations read from the input dataset.

[Tom]

1. What does the retain statement do here for this code?

It prevents FirstSnow from being reset to missing at the start of the next iteration.  So the value is "retained" from iteration to iteration of the data step only changing when a new value is assigned.

2. Why is an If-Then-Do statement necessary here? Why do those assignment statements have to go under the last.Code=1 part?

It is actually NOT needed, only the OUTPUT statement needs to be executed conditionally.  Try replacing the IF/THEN/DO/END block with these statement instead.

        LastSnow=Date;
        WinterLengthWeeks=intck('week', FirstSnow, LastSnow, 'c');
        ProjectedFirstSnow=intnx('year', FirstSnow, 1, 'same');
        if last.Code then output;

But the IF/THEN/DO block makes the logic clearer to humans and a small amount faster since the assignments statements execute only once per value of CODE.

3. I introduced all the missing parts from the answer code into my code one by one. As soon as I included "output;", the table went from about 140+ rows to only 4. I don't understand how output did this here.

The OUTPUT statement tells the data step to write an observation to the output dataset(s).  If your step does NOT have an explicit OUTPUT statement then SAS will imply one that will execute every iteration that reaches the end of the step.   

With an OUTPUT statement that executes only once per value of CODE then only one observation per value of CODE will be written out. 

Without an explicit OUTPUT statement in that code then every observation read in that reaches the end of the data step is written out.   This means you do not even need an OUTPUT statement. Instead you can use a subsetting IF so that only the iterations where you want an observation written makes it to the end of the data step.

        if last.Code ;
        LastSnow=Date;
        WinterLengthWeeks=intck('week', FirstSnow, LastSnow, 'c');
        ProjectedFirstSnow=intnx('year', FirstSnow, 1, 'same');

Note that the IF conditions do not need to contain the =1 part you added in your post.  The FIRST. and LAST. flags are assigned values of either 1  or 0 .  SAS treats any number that is not zero or missing as TRUE.  So the code reads more smoothly without the equality test.  "if last code" versus "if last code flag's value is one".

[Kurt_Bremser]

My solution would have been shorter:

data snowforecast;
set winter2015_2016;
retain FirstSnow;
by Code;
if first.Code then FirstSnow=Date;
if last.Code;
LastSnow=Date;
WinterLengthWeeks=intck('week', FirstSnow, LastSnow, 'c');
ProjectedFirstSnow=intnx('year', FirstSnow, 1, 'same');
format FirstSnow LastSnow ProjectedFirstSnow date7.;
drop Snow Date;	
run;

The code will read all observations, but keep only the value of the first observation in a group in variable firstsnow.

Then, it will only proceed further if the last observation in a group has been read. All following statements (including the implicit OUTPUT at the end of each step iteration) are "jumped over" in all other cases.

This particular form of an IF statement is called "Subsetting IF"; if the condition is not met, the next iteration of the DATA step is immediately started.

[sfletc30]

But format and drop are not skipped over because they are compile-time statements that aren't affected by the OUTPUT statement. Is this right?

[Tom]

---

@sfletc30 wrote:
But format and drop are not skipped over because they are compile-time statements that aren't affected by the OUTPUT statement. Is this right?

---

Right.  They effect the definition of the dataset(s) that is (are) created by the step.  What formats are attached to the variables. Which variable are included.

But the FORMAT statement's location in the data step can vary its impact because that can change the order that the data step compiler sees the variable names.  So that will impact the order that the variables are stored in the dataset. And it can impact how it decides to define those variables.

For example if the first place that the compiler sees the name of the variable is in a FORMAT statement it will guess that you meant to define the variable with a TYPE that is compatible with the type of format that it is being told to attach.  And when that type is CHARACTER it will base is choice of the LENGTH to use to store the variable based on the width of the format that you told it to attach.