I am having the same issue.  I try to be respectful of those who know more than I do about these issues, but I keep coming back to the "gremlins did it" explanation proposed back in 1978 in the seminal work "Towards a Unified Theory of Computer Glitches, SNAFUs, and FUBARs: The Gremlin Theory of Why Bad Things Happen to Good People and Their Computers and IT Infrastructures" (Journal of Zoological Cybernetics, 6:3 (April 1978), pp. 27-45). The "gremlins did it" explanation still seems to have as much explanatory power as more recent attempts to explain paranormal cybernetic anomalies - especially if gremlins are posited to have some of the characteristics of Higgs-Boson particles as suggested in some more recent work. Also see "The catastrophic Mysterious Nefarious Actions of Gremlins, Golems, Goblins and Tribbles in Cyberspace" in Roy Bradberry (ed.), Mysteries of Cyberspace, NY, Random Data House, 2020, pp.179-228. ... View more

I have downloaded the "language" for SAS that creates the color coding for SAS programs.  The core piece of the EG project log text editor is the "collapsibility/expandibility" feature.  It would be very nice to have that in logs that have to be exported after the space available in EG for project logs has been reached.  Does anyone know of such a feature or text editor? ... View more

I am talking about the project log -- not the logs from the program nodes. In addition to color coding, the project logs "collapse".  That is, each run has a heading indicating the node that was run and the exact date and time it was run.  Each heading can be expanded to show the log from that run or collapsed so that one can quickly scroll through the log.   ... View more

Re Maxims 40  and 42:   It is the job of analysts to consult with clients and not just do what they ask.  They do not always know what they want or need -- or at least cannot always articulate this in a way that is meaningful to analysts.  When consulting with them it often becomes apparent that what they asked for is not exactly what they need.   I often (correctly) anticipate that as soon as I give them what they need they will ask the next question and they will ask for a "deeper dive" into the data.  I have therefore found that it is often efficient to pull more data than is required for the immediate task -- or to at least include the code to do so even if I "comment it out".  Doing this makes for ugly logs (see Maxim 25 above) but for analytic work that is not in production I would still recommend this practice. ... View more

Many of us also work across platforms, for example querying a DB2, Oracle, Netezza or Terradata database and writing our SAS dataset to the unix server on which SAS is installed. In my experience, (usually small but sometimes fateful) computer precision errors may occur simply because data are transferred across platforms. SAS has tools/functions to deal with this (e.g. ROUND and FUZZ). But we need to be aware of the issue. In my work, an issue that occurs frequently is the expectation that when I sum values that should result in 0 the actual sum may be 0.00000000001 (or similar). When I subsequently filter on "WHERE VAR0 = 0" I get unexpected results. Benjamin (Benjy) Ben-Baruch Business Intelligence and Marketing Analytics Harry and David, LLC www.HarryandDavid.com<> ... View more

This may in fact be a good "work-around" solution,     I would think that the "better" "work-around solution" would be to allow for the exclusion of strata with no sample selected in the PROC SURVEYSELECT output dataset and then merge these records back in.  Just as many steps -- but no danger of misleading values for sample-weights and selection probabilities in the final dataset.   However, I was hoping for a "real" solution that requires neither "gerrymandering" the data with an additional data step or having to merge unsampled strata back in.  I guess maybe I will just have to go away singing "You can't always get what you want." ... View more

"Best practices" differ slightly depending upon the type of work one is doing. Developing "production code" is different from developing analyses. And analysis is different from reporting. (Most people with the title of "Analyst" actually don't do any analysis at all, rather reporting of data in ways that assist people in understanding them. They are "data clerks" but unfortunately the job title "clerk" like "secretary" has been devalued in our society so people are given titles that obscure what they really do. And just as unfortunately, by not valuing "data clerking" and reporting in their own right, this work is also denigrated. An unfortunate additional result of this is that the terms "analysis" and "data mining" have become meaningless. Analysis means any work that reports numbers in a different format or aggregation from the way they are stored. Data mining now means any query into a "large dataset" or any query involving two or more datasets. The goal of data mining has become shovelling out the dirt rather than finding the diamonds-in-the-rough.) The main point is that best practices for 1. writing production code, 2. data reporting and data clerking, and 3. data analysis and data mining differ slightly. Sometimes these differences were noted in the maxims above, but this needs to be more explicitly acknowledged. ... View more