Hi,
These are really EXCELLENT questions, especially for a beginner. I wish I had asked these questions earlier in my SAS programming. It took me a few years to really understand the DATA step.
Importantly, your general understanding is wrong:
My general understanding of `set lib.table` is that it reads the _N_'th observation from the given table where _N_ is the PDV's internal counter.
The DATA step is an implied loop. _N_ is simply a counter of the number of times the loop has executed. There is no causal relationship / dependency between _N_ and the SET statement. When the SET statement executes, it reads the next record from a data set. What does "next record" mean? The SET statement has its own pointer which tracks which record to read from a data set. There is often a correlation between _N_ and the SET statement because many steps happen to execute the SET statement once for each iteration of the DATA step. But that is just a correlation.
The below step iterates 20 times. The log shows that on each iteration of the DATA step, one record is read (and output):
data want ;
put "Top of loop " _N_= ;
set sashelp.class ;
put "Bottom of loop " _N_= Name= ;
run ;
On the 20th iteration of the loop, the SET statement executes. Because there is no next record to read, it hits the end of file marker and that causes the DATA step to stop executing. Note that the "Bottom of loop" PUT statement does not execute on the 20th iteration, because the step stopped executing when the SET statement executed.
The below step iterates only four times, but also reads and outputs all 19 records from sashelp.class.
data want ;
put "Top of loop " _N_= ;
do i=1 to 5 ;
set sashelp.class ;
output ;
put "Inside DO loop " _N_= i= Name= ;
end ;
put "Bottom of loop " _N_= Name= /;
run ;
On the first iteration of the DATA step (_N_=1), the explicit do loop iterates 5 times, so the SET statement reads the first 5 records. On the second iteration of the DATA step (_N_=2), the explicit do loop iterates 5 times, so the SET statement reads records 6-10. On the 4th iteration of the DATA step (_N_=4), the explicit do loop iterates 5 times. On i=1 to i=4, the SET statement reads records 16-19. When i=5, the SET statement tries to read the next record, it hits the end of file marker, and the DATA step stops executing immediately.
In answer to other questions:
1. The SET statement does NOT change its meaning inside of a DO loop. That would be chaos if it did. If this is not clear, please post an example where you think this is happening.
2. Each SET statement that is reading from a data set uses its own internal pointer to keep track of which record to read. If there are two SET statements in a step, each still has its own pointer, and the two SET statements are independent of each other.
To work through your second example (sorry, I'm out of time to write more), I would recommend you add some PUT statements, something like below, and remember that each SET statement has it's own pointer, the two SET statements are independent of each other, and there is only one PDV for the step.
data test2;
length name $13 ;
put "Top of DATA step " (_N_ Var Name Rate)(=) ;
set test;
put "Before DO loop " (_N_ Var Name Rate)(=) ;
do var=1 to 2;
put "Top of DO loop " (_N_ Var Name Rate)(=) ;
set test;
output;
put "Bottom of DO loop " (_N_ Var Name Rate)(=) ;
end;
put "Bottom of DATA step " (_N_ Var Name Rate)(=) /;
run;
If you can't figure out what is happening, respond with more questions. I or others will happily explain more. These are good questions, and will raise a lot of issues critical to understanding DATA step programming.
The Boston Area SAS Users Group is hosting
free webinars!
Next up:
Joe Madden & Joseph Henry present Putting Power into the Hands of the Programmer with SAS Viya Workbench on Wednesday Nov 6.
Register now at
https://www.basug.org/events.
