Hi, If your original data is stored in a XLSX file then you can use the new %xlsx2sas macro to transform it to SAS data set without going through CSV format. The %xlsx2sas macro supports NLS encoding (UTF-8). Demo download & free trial at: www.bixforsas.com ... View more

Hi , SPDE is included with SAS BASE and it does not require a separate license. I don't see why scanning your 3 TB once takes 3 days..? I have experienced scanning 1.3 Billions X 70 variables sas data set stored as a SPDE data set on a LINUX box + disk storage within 25 minutes... Gady ... View more

In order to solve this size of a problem in a reasonable time you'll need to use proc means (as you have been using) (Hashing with only 8 GB of RAM will run out of memory immediately). Try to increase memory size known to the SAS session ("Memsize" and "Sumsize" invocation options). Try to use SPDE engine with array disk storage. Another approach is to split the 3TB data set into many small data sets by the values of class variables equipment and date and then use proc means to get the statistics for each of the small data sets. You'll need a SAS Macro knowledge to this "trick" and a table of all possible combinations of "equipment" and "date" in you 3 TB data set. Then in one datastep you can split the file into smaller parts. Then execute proc means for each of the parts. At the end of the job you can append the final results back again into one data set. Gady Kotler www.bixforsas.com ... View more