DATA Step, Macro, Functions and more

Making work queues of equal sizes

Accepted Solution Solved
Reply
New Contributor
Posts: 3
Accepted Solution

Making work queues of equal sizes

Explanation:

SAS 9.4 m4


I've got about 100 pieces of work to accomplish. I know what the average run time is for each piece of work. I have only 4 connections to my database and I want to make the work run optimally. I tried proc hpbin but it does not do what I really want, but does give you an idea of my data.

proc hpbin data=work.hbin_data output=work.binout numbin=4 bucket;
id job_nm jobwall_p95;
input jobwall_p95;
run;

I then write a proc summary to see how it did and I have 4 queues, but they are not close to being equal as that not what HPBIN does.

BIN                  JOBWALL_P95_Sum             JOBWALL_P95_N
1                        378.66666667                             77
2                        129.81666667                               3
3                        173.71666667                               2
4                        132.06666667                               1

 

So, HPBIN makes 4 bins with the first one that runs small jobs and takes 378 seconds

Please help?


Accepted Solutions
Solution
‎10-09-2017 02:48 PM
Super User
Posts: 13,293

Re: Making work queues of equal sizes

[ Edited ]

DASDguy wrote:

My problem is that some jobs take .06 seconds and other take over 100. So, there is quite some disparity there. I don't have SAS/OR. Smiley Sad

 

 


On second though the mod approach, will tend to have the 4th bin run longer.

Consider:

data junk;
   incr= 1;
   do i = 1 to 100;
      bin+incr;
      if bin=5 then do;
         incr= -1;
         bin=4;
      end;
      if bin=0 then do;
         incr= 1;
         bin=1;
      end;
      output;
   end;
run;

proc means data=junk nway sum;
   class bin;
   var i;
run;

 

 

 

View solution in original post


All Replies
Super User
Posts: 13,293

Re: Making work queues of equal sizes

If the issue is to assign your jobs into 4 bins then sort the data by the runtime and then apply a bin number using MOD function based on order in the data set.

The below example creates 100 records in sorted order. Think of the value as runtime order. The mod function would return 0 to 3 with mod 4, so I add 1 in case 0 isn't an acceptable bin number. You could sum the runtimes with the bin as a class variable to see the expected result.

 

data have;
   do i=1 to 100;
      output;
   end;
run;

data want;
   set have;
   bin = mod(_n_,4)+1;
run;

If you have SAS/OR licensed I suspect that you could get a better balance but I suspect  this approach may be "good enough" unless you have one or two jobs that run orders of magnitude longer than most of the others.

 

New Contributor
Posts: 3

Re: Making work queues of equal sizes

My problem is that some jobs take .06 seconds and other take over 100. So, there is quite some disparity there. I don't have SAS/OR. Smiley Sad

 

 

Solution
‎10-09-2017 02:48 PM
Super User
Posts: 13,293

Re: Making work queues of equal sizes

[ Edited ]

DASDguy wrote:

My problem is that some jobs take .06 seconds and other take over 100. So, there is quite some disparity there. I don't have SAS/OR. Smiley Sad

 

 


On second though the mod approach, will tend to have the 4th bin run longer.

Consider:

data junk;
   incr= 1;
   do i = 1 to 100;
      bin+incr;
      if bin=5 then do;
         incr= -1;
         bin=4;
      end;
      if bin=0 then do;
         incr= 1;
         bin=1;
      end;
      output;
   end;
run;

proc means data=junk nway sum;
   class bin;
   var i;
run;

 

 

 

New Contributor
Posts: 3

Re: Making work queues of equal sizes

The method of "snake drafting" is good enough for my purposes. 

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 4 replies
  • 115 views
  • 0 likes
  • 2 in conversation