And that's it Thank you Respect ... View more

You can simply do data b; set a; output; datetime + '04:00't; output; datetime + '04:00't; output; datetime + '04:00't; output; datetime + '04:00't; output; run; But a code with a DO loop will look much cleaner, and be easier to adapt to other requiremments. ... View more

If you give %SYSFUNC() at least two arguments then it will not cause the COUNTW() function to issue that error. 937 %let List =; 938 %let N_List = %sysfunc(countw(&List,)); 939 %put &=n_list; N_LIST=0 But you really should be telling COUNTW() what delimiters to use.  For example a space. %let N_List = %sysfunc(countw(&List,%str( ))); Otherwise it uses multiple characters.   the default delimiters depend on whether your computer uses ASCII or EBCDIC characters. If your computer uses ASCII characters, then the default delimiters are as follows: blank ! $ % & ( ) * + , - . / ; < ^ | In ASCII environments that do not contain the ^ character, the SCAN function uses the ~ character instead. If your computer uses EBCDIC characters, then the default delimiters are as follows: blank ! $ % & ( ) * + , - . / ; < ¬ | ¢ To simply test if the sting is empty I normally just test for non-zero length. %if %length(&list) %then ... ... View more

@Kurt_Bremser wrote: replace spinning metal with state of the art (SSD) 🙂 @Kurt_Bremser wrote: organize your processes, so they run in sequence It would cost me too much work and the effect would be unsatisfactory. ... View more

> This is not optimal in terms of disk space, but in terms of computing speed, yes I doubt it, and as you can see I am not the only one. And we haven't mentioned the time needed to create and delete these files.   Anyway, it seems you have two solutions: - Implement our suggestions and have larger files. Maybe use 2 variables instead of 3?.Or even 1. - Keep your method, In this case, you create 200,00 files, process them, and then do the same for the other half. .     ... View more

What about maintaining a table - maybe by excel - containing few columns: 1) input dataset name  2) condition for sub-setting relevant observations 3) yes/no flag - should this row participate in current run   then transfer the data from the table to macro variables to be used with macro %loop or alternatively use call execute to submit the code to run adapted to current row, or even generate code like @gamotte posted: data want set <input1>(where=(<condition1>)) <input2>(where=(<condition2>)) .... ; run; ... View more

All of the advice that you have received so far is good advice.   Sometimes, however, depending on your company's policies, certain things like editing config files aren't always possible.   One other option is to run your program in "batch" mode (instead of interactive).  The command varies by environment (Windows, Unix, Mainframe, etc.).  Here's an example from Windows where I've created a little .bat file which, when double-clicked, launches my SAS job in batch mode with all the parameters that I have specified: set env=production ::set env=development sas -set environment %env% -set lPwd_PASSWORD XXXXXXXX -memsize 32G -work N:\Work -logparm "rollover=session write=immediate" -log C:\Users\XXXXXXXX\Documents\SAS\Pgm\Test\Hive_Grid\logs\Join_Members-tblMMR0323_Restated_MONO_#Y-#m-#d_#H-#M-#s_#p.log -sysin C:\Users\XXXXXXXX\Documents\SAS\Pgm\Test\Hive_Grid\Join_Members-tblMMR0323_Restated_MONO.sas -nostatuswin -noterminal -nosplash -noicon pause Some of the parameters I'm using in the above example: Work.  I'm assigning Work to a drive that, in this case, has more space because I have some big files to work with. Memsize:  Again, I'm using big files, so I'm increasing my memory allocation. Logparm:  I'm using "immediate" which means that the log is written too as soon as SAS has log output.  Otherwise SAS will buffer the writing to the log.  When buffered, you can lose part of the log if the job crashes, and, if you have long running steps, it can take forever for enough log data to get into the buffer to the point where SAS actually writes it to the log.  Caution:  "Immediate" is less efficient.  "Immediate" is a great option for debugging, but once a job is stable, "Buffered" is typically going to give best performance. Log:  I'm using a series of log directives which tell SAS how to dynamically name the log file.  Here, I'm adding a system generated data time stamp and the Process ID to the name of the log file. You don't have to code all of these parameters, Logparm for example is not essential; this is just an example.   All that to say, sometimes it's better to submit via batch when you need to have a little more control over parameters that must be set at start up.   Jim     ... View more

@makset  Your question is way to vague and generic to propose anything else than reading/books. So if it's about dealing with large data sets then you might want to Google for SAS topics dealing with performance - and there's lot about this out there. Also spending time reading SAS® 9.4 Language Reference: Concepts, Sixth Edition is likely valuable for you as it explains how SAS actually works and how it processes data.   About SAS hashes: They get loaded into memory with fully expanded column length so likely not suitable for your huge tables unless you've got actually the required memory available. ... View more

OK thank you ... View more

@Reeza wrote: 1GB isn't a large data set for SAS in most cases. What makes you think it is? A hundred files of 1GB is a lot, especially if the application is to work more or less in real time (10 seconds is a lot, 1 min is forever). @Reeza wrote: Are you trying to get 40 observations or a window of time (30 minutes it seems) relative to the time in table 1? I think its the latter but that isn't clear to me. 40 observation. data A; informat datetime datetime21.; input info $7. datetime Nr ; format datetime datetime21.; cards; GC_5_0 30APR2020:00:30:00 1 GC_5_2 30APR2020:00:40:00 2 GC_50_4 24NOV2010:21:00:00 3 GC_5_1 28OCT2003:13:35:00 4 GC_8_1 01AUG2016:11:20:00 5 GC_5_3 09AUG2016:18:10:00 6 GC_10_2 06SEP2019:06:40:00 7 GC_5_0 30APR2020:00:39:30 8 GC_15_2 27NOV2006:09:45:00 9 GC_5_2 30APR2020:00:17:00 10 run; data GC_5_0; set GC_5_0; nrdb = _n_; run; data GC_5_2; set GC_5_2; nrdb = _n_; run; %macro test(); * for simplicity; data AA; set A; if info = 'GC_5_0' | info = 'GC_5_2'; run; proc sort data = AA out = AA_count nodupkey; key info; run; * Obs_count variable needed for the loop; data _NULL_; if 0 then set AA_count nobs = n; call symputx('Obs_count',n); stop; run; %put no. of observations = &Obs_count; * I need to search by number of observations, not by datetime; data _null_; if _n_ = 1 then do; declare hash H (multidata:"y"); H.definekey("info"); H.defineData( "datetime","Nr"); H.definedone(); end; do until (last.info); set AA; by info; H.add(); end; H.output (dataset: catx("_", "AA_", info)); H.clear(); run; %do i = 1 %to &Obs_count %by 1; * extracting the source file name; data _NULL_; set AA_count; if _n_ = &i then do; call symputx('info',info); end; run; %put &info; data aa__&info; set aa__&info; if _n_ = 1 then do; declare hash H1 (dataset:"&info"); H1.definekey("datetime"); H1.definedata("nrdb"); H1.definedone(); end; if H1.find() = 0 then nrdb = nrdb; run; * for simplicity, remove undiscovered data; data aa__&info; set aa__&info; if nrdb ne .; run; data wont_&i; set aa__&info; if _n_ = 1 then do; declare hash H (dataset:"&info"); H.definekey("nrdb"); H.definedata("datetime","high_temp","low_temp","end_temp"); H.definedone(); end; do nrdb = nrdb - 20 to nrdb + 19 by 1; if H.find() = 0 then output; high_temp = high_temp; low_temp = low_temp; end_temp = end_temp; end; run; * merging partial tables into the final table; %if &i = 1 %then %do; data wont; set wont_&i (drop = nrdb); run; %end; %else %do; data wont; set wont wont_&i (drop = nrdb); run; %end; %end; proc sort data = wont out = wont; key nr; run; %mend test; %test; Maybe it can be done faster and smarter. Best regards   ... View more

@makset  - With SAS's latest architecture SAS Viya, in-memory processing is the default methodology and it happens without you having to do any special coding. ... View more

Yes, the "START SAS" command works, inasmuch as it starts a process. But your subtask is now START, which will return as soon as the SAS task is started. ... View more