Hash objects use your available memory, and that is limited by the MEMSIZE option; the first opinion is correct.
When you have hash objects in production jobs that come close to memory limits, you need to constantly monitor the memory usage during such DATA steps (read the logs!) so you avoid unexpected crashes.
... View more
The first question that arises: why do you need to do this?
This dataset looks very much like one of the dreaded "wide" datasets which always cause trouble and are much harder to handle than "long" datasets. This also smells like data hidden in structure.
The only fully automated way to handle this which comes to my mind is a double TRANSPOSE with a modification of the long dataset:
data _temp_;
input x1 x2 x3 x4;
n = _N_;
datalines;
1 2 3 4
;
proc transpose
data=_temp_
out=long
;
by n;
var x:;
run;
data long2;
set long;
substr(_name_,1,1) = "y";
run;
proc transpose
data=long2
out=want (drop=_name_)
;
by n;
var col1;
id _name_;
run;
Note that, for the second TRANSPOSE to work correctly, you need a variable (or variables) to identify the original observation and used in the BY.
Let us know how you arrive in this situation in the first place.
... View more
For working, you always use internal disks connected to the main bus via SATA. The external USB connections are good for backup and similar, but not for real heavy load working.
... View more
Either you have a completely inadequate disk (read: from the 90's), or you have other processes congesting your computer.
Process monitor from Microsoft Sysinternals can be handy.
... View more
TBH, a large part of my accepted solutions have been rehashes of other solutions/posts, and "translation" of SAS documentation for the novice user. In short, I was a better search engine.
With the advent of AI-supported searches in Google and others, these have taken over that particular role, and so the people coming here for help find answers much easier before they post a new thread asking the same old, same old.
And they will be pointed towards relevant documentation also, including examples. Knowing where (and how) to look for relevant documentation has always been a daunting task, given the sheer size of SAS doc.
... View more
This
filename out zip 'Batchfile 7.6.zip';
is your problem.
You did not use a fully qualified path name, so SAS tries to write to its current working directory, which is usually the place where the script file
WorkspaceServer.sh
is located, and where you do not have write permission.
Use a file name that includes the complete path starting at root (like you did for indir) and pointing to your home directory. You can also use the tilde shortcut for this:
filename out zip '~/Batchfile 7.6.zip';
... View more
Something unbalanced (unbalanced quotes, macro definition without a valid %MEND, ...) has caused your SAS session to become unstable. Start a new session and rerun your code, paying attention to the above.
... View more
One must make such transformations (assignment of formats, extraction of sub-values etc.) when the dataset is prepared for analysis. This is a job for your data engineers.
... View more
You tried to use a library reference (libname) as infile in PROC IMPORT, which cannot work. Either use a worksheet like a SAS dataset in your Excel library, or use the correct physical filename in the DATAFILE= option of the PROC IMPORT statement.
... View more
@sfletc30 wrote:
Hey so just a followup question for number 3. So I understand what the code is saying now. If last.Code=1, then do these things (which includes writing to the output table with the OUTPUT statement). And last.Code=1 only exists four times in the dataset. So that explains why the rows went from around 140 to only 4.
But why do I have to explicitly tell SAS this? Why doesn't the implicit OUTPUT statement at the end of the code create the four rows that the explicit one does?
Because the implicit OUTPUT writes an observation for every iteration of the DATA step, unless the iteration is cut short by a subsetting IF. And the number of iterations is (in your case) determined by the observations read from the input dataset.
... View more
