Don, I remember those "yeah, but"s quite well and rather fondly. To what you've butted here, I have but a buttless yeah.   Kind regards Paul D  ... View more

Hey @yabwon / @ChrisNZ / @DonH:   Forming a CP is a matter of reading X one record at a time and pairing it with each record from Y sequentially. Sequential read is a not POINT='s forte, nor it is a hash object's forte, as the iterator has a significant underlying software overhead. Both rather excel at searching for a given obs number or key-value. The most natural way of speeding up repeated sequential scans is a temp array because it reads from memory and does it fast. Hence, for example - using Bart's test data (and not worrying about hardcoding):   data test_arr ; array ya [10000] _temporary_ ; array y1a [10000] _temporary_ ; array y2a [10000] _temporary_ ; array y3a [10000] $1 _temporary_ ; if _n_ = 1 then do until (z) ; set y end = z; ya [_n_] = y ; y1a[_n_] = y1 ; y2a[_n_] = y2 ; y3a[_n_] = y3 ; end ; set x ; do _n_ = 1 to 10000 ; y = ya [_n_] ; y1 = y1a[_n_] ; y2 = y2a[_n_] ; y3 = y3a[_n_] ; output ; end ; run ; actually outperforms SQL by about 20 percent in my tests on the same laptop I'm typing this. Which kind of makes me ideate that SQL does kind of the same sort of thing behind-the-scenes.   Kind regards Paul D.           ... View more

@novinosrin: Thank you for the kind words, Naveen.  ... View more

@sdaniels429  ... set fill point = p ; means "read observation # P". In this case, the FILL data set has but one record we want to read every time the name is Smith, so we need P=1. RETAIN is used to populate P with 1 for the duration of the step without the need to assign P=1 when name is Smith. Note that we cannot code just point=1 since the option cannot accept a literal; it only accepts a variable. BTW, this variable (in this case, P) is auto-dropped.    Now when set fill point = p is executed, the values of the variables in FILL overwrite the values of the corresponding variables in the PDV, quod erat faciendum. SASFILE isn't a function but a statement placing FILL into memory just to make SET FILL execute faster than reading from disk.    Kind regards Paul D. ... View more

@ChrisNZ: Oops, the right Chris ... sorry! To reiterate, a fantastic hash size calc.   Kind regards Paul D.  ... View more

@PeterClemmensen:    The fact that item_size returns its result in bytes is not system-specific. But the result is: For example, item_size is generally shorter under 32-bit Windows than on 64-bit. It also depends on the key or data type (numeric or character).         As to the reason why item_size may be longer than the total number of bytes in the key and data portions, my understanding is that it accounts for extra memory needed for internal bookkeeping.        @chrisz has written a fantastic macro routine covering all item_size eventualities for practically any combination of the length and scalar data types (i.e. character or numeric) and formulated item_size rules depending on these factors. I haven't seen any exceptions from these rules. The routine does not cover non-scalar types, such as the pointers to hash objects and/or iterators that can be stored in the data portion of a "hash-of-hashes" hash table. My limited experimentation into it has shown that the effect of having such piece of data in the data portion is equivalent to adding a (scalar) numeric hash variable.        Kind regards Paul D. ... View more

@DonH ... Thanks and fair enough ... but how do we qualify the variables that have no "intended usage" to begin with? For example, at times I use first._n_ and last._n_ as utility variables since the are perfectly fine numeric variables set to 0 at compile and auto-dropped from the output. Actually, first./last. can be attached in this manner to _error_, _iorc_ and any other variable. Even if it's not in the PDV - and even if it is a non-scalar variable in the PDV! Watch this: data _null_ ; dcl hash h ; dcl hiter i ; put first.h= last.h= first.i= last.i= first.var= last.var= ; run ; A curious thing is that though PUT shows all of them =0, you get the "unitialized" messages, too 😀. Looks as though the compiler sees no association of first.X / last.X with X at all unless X is part of BY - in which case, by the way, it sets them to 1. Proof: data _null_ ; put first.name= last.name= ; stop ; set sashelp.class ; by name ; run ; Kind regards Paul D. ... View more

Hi @novinosrin: Yup, that's the logic, to a tee. Thanks for another plug!    Kind regards Paul D.  ... View more