Well if I was to spit up a 100 line dataset into subsets of 10 lines each , then split a 1000 line data set into subsets of ten lines each. That would give me different numbers of subsets would it not? ... View more

Ok, I want to split some datasets into a number of smaller sub-sets. The number of sub-sets will vary depending on the size of the input dataset.   I then want to set up a loop for each  sub-set and process it. I don't know in advance how many sub-sets there will be, only after the splitting has happened. In psuedocode,   Split dataset1     Produces  5 sub-sets say    for i = 1 to 5        process subset_i     Split dataset2    Produces  100 sub-sets say    for x = 1 to 100        process subset_x   ... View more

Can you create a variable list "on the fly". Something like (forgive the syntax as I'm a newbie)   %let myvar=;   %do i 1 to 5; myvar = myvar || "table_" &i.;   %let varlist = &myvar.; ... View more

We are doing some numerical and frequency analysis a series of big datasets. A small data set would be 2.5 Gb and going up to 500Gb. When I tried our analysis code on the one of the small datasets I got a memory error   Table XYZ (Number 1) Frequency Analysis began at Runtime 10:36:48. Rundate 11/07/2019. Table XYZ (Number 1) has 537 Fields for Frequency Analysis ERROR: PROC SUMMARY was terminated prematurely due to a memory shortage. Adjusting MEMSIZE and/or SUMSIZE may allow normal        processing to complete. ERROR: The SAS System stopped processing this step because of insufficient memory. WARNING: The data set WORK.SUMM1 may be incomplete.  When this step was stopped there were 0 observations and 2 variables.   So one thought is to split up the larger datasets into smaller ones and do the analysis on each of the sub-sets ... View more

Something like that. There are multiple datasets as input. Each input dataset is to be split up depending on its size.   Input dataset1 might end up being split up into 5 sub datasets, input dataset2 might be split up into 10 sub-datasets.   At the end of each input dataset splitting I'll know how many sub-sets it was split into. How do I then cycle through and get the name of each sub-set into a variable that I can use to process it. ... View more

Let's say I had  large datasets that I wanted to divide up into smaller datasets and then process each new sub-set of data. On one run the sub-division might give me 3 sub-sets, TABLE_1, TABLE_2, TABLE_3 . On another run it might give me 100 sub-sets, TABLE_1-100  etc ... ... View more