11-07-2014 11:30 AM
I am working on a large data set for first time, and I have to choose couple variables from the given long list of given thousands of variables . Also dataset has around 850,000 observations. Every time I run two or three data steps in sas , sas tells me output window full. (NOTE: SAS IN WINDOW ENVIRONMENT), I know unix does better, but don't have that.
Question1: how to deal with this issue. How to clear output window. I tried ODS HTML CLOSE;ODS HTML;
but when sas output window is full it does not allow me to run these commands. If you have other good ideas to share on big data management that would be helpful.
Second question: How to cut short my data set to try to get required data , I mean I want to try my commands on a small subset of data to see if they work properly , so that later I can apply them on entire data set to get my required data to carry out the analysis.
Thanks for your time
11-07-2014 11:42 AM
ods listing close; If output is going to window.
to select a limited number of records (observations) you can use the dataset option OBS= anytime you are reading a dataset:
proc freq data=example(obs=100); to use 100 records from the dataset example.
set have (obs=100 ); to read 100 records into the new data set.
If you only want to have a few variables out of a long list then you can use the KEEP= option
set have (keep= varname1 anothervarname thatvar);
You must be generating output to fill up the window. What are you doing to write to the output window in a data step?
11-07-2014 12:06 PM
Depending on your install SAS will default to sending some types of output to either the Listing destination (Output window) or HTML (Results window) and possibly both. If you are running DATASTEP code and having the Output window fill up, then your code is being told to write output, possibly with a FILE PRINT statement. You might want to post an example of the code that is causing the issue.
And since datastep code will generate an output data set unless using the special set _NULL_, I really am curious what you're doing currently.
11-08-2014 07:05 AM
data want (keep = a b c );
set have (obs=200);
if you want to create a random sample then
data want(keep=a b c drop=i );
do i =1 to 5000 by 10;
11-08-2014 11:55 AM
In the ancient time when big data meant you had a lot of cartons with punch cards the interface was designed for humand access.
- sysin where you put your cards in.
- print that print on real paper for the results
- saslog The messages what your program did.
These basics are still there. As it should human readable there has been limits set on the log and Print.
When your ouput is going to a file you can turn the results off. SAS(R) 9.4 Output Delivery System: User's Guide, Third Edition
As you are under a windows license with DMS
There are al lot of windows-specific options. SAS(R) 9.4 Companion for Windows, Third Edition
But the sizing of the log is set to 9999 for all OS types. SAS(R) 9.4 System Options: Reference, Third Edition dmslogsize
the output (results) can also be set. SAS(R) 9.4 System Options: Reference, Third Edition DMSOUT 2G of lines is an awful lot
You could cleanup before your are going to run.