topic how to run ds2 actionset runmodel on multiple threads in SAS Programming

how to run ds2 actionset runmodel on multiple threads

acordes — Tue, 03 May 2022 10:06:36 GMT

It takes a long time to score and once finished in the log a note states that only one thread was used.

proc cas;
      loadactionset "DS2";
      runmodel /
      modelname="Forest_recovery_maig2_tarde",
      modeltable={caslib="DNA", name="sas_model_table" },
      table = {caslib="public", name="recovery_candidates2" },
      casout={caslib="busidev", name="ac_scored"};
run;

Re: how to run ds2 actionset runmodel on multiple threads

Rick_SAS — Tue, 03 May 2022 10:17:14 GMT

The documentation for the DS2.runModel action

SAS Help Center: DS2 Details

includes this caution: "It is recommended that the DS2 code that is run with the runModel action include a THREAD program. This allows the DS2 code to run as a DS2 parallel program and therefore leverage the full set of worker nodes on the CAS server."

You can see the DS2 documentation for more information about how to use the THREAD/ENDTRHEAD block for parallel processing: SAS Help Center: THREAD Statement

Re: how to run ds2 actionset runmodel on multiple threads

acordes — Tue, 03 May 2022 11:54:35 GMT

I don't understand what I'm doing... I copy/paste from the link you shared.

Fact is that I don't succeed in getting the table PUBLIC.RECOVERY_CANDIDATES2 scored.

The code without thread works but is very slow.

The thread option is resolved correctly but I don't know what the runmodel statement should look like.

here is my failed intent.

proc cas;
  loadactionset "modelPublishing";
  publishModel  /                                       /*5*/
  modelName="Forest_recovery_maig2_tarde",
  modelTable={caslib="DNA", name="sas_model_table" },
  program= "
     thread cars_thd / overwrite=yes;
       method run();
         set PUBLIC.RECOVERY_CANDIDATES2;
/*          if (msrp > 100000) then do; */
/*            put make= model= msrp=; */
/*            output; */
/*          end; */
       end;
     endthread;
     data busidev.cars_luxury /overwrite=yes;
       dcl thread cars_thd t;
       method run();
         set from t threads=8;
       end;
     enddata;";
  run;

      loadactionset "DS2";
      runmodel /
      modelname="Forest_recovery_maig2_tarde",
      modeltable={caslib="DNA", name="sas_model_table" }
/*       table = {caslib="public", name="recovery_candidates2" }, */
      table = {caslib="busidev", name="cars_luxury" },
      casout={caslib="busidev", name="ac_scored2"};
run;

Re: how to run ds2 actionset runmodel on multiple threads

acordes — Tue, 03 May 2022 12:12:35 GMT

Ok, I'm starting to understand a little bit.

I overwrite my model Forest_recovery_maig2_tarde with the publishing option and then nothing happens because I do not define a 'model' in the program statement...

So I need to copy/paste the method run() block from the model studio where I trained the model, right?

Or how do I run the EXISTING model in multiple threads without the modelpublishing block?

Re: how to run ds2 actionset runmodel on multiple threads

acordes — Tue, 03 May 2022 12:21:38 GMT

@Rick_SAS I can use alternatively the proc astore and it works.

But I want to use the ds2 actionset because I want to score within the IMLactionset calling 'submit'.

It works, but it's slow if it's not multithread.

Re: how to run ds2 actionset runmodel on multiple threads

Rick_SAS — Tue, 03 May 2022 12:40:25 GMT

I do not have any experience doing what you are attempting to do. However, when I want to write a complicated program, it always goes better when I slowly study and implement each individual piece. Start with a regular DS2 program that has a THREAD statement. Get it working in multiple threads. Move on to creating and publishing a model, and so forth.

Good luck!

Re: how to run ds2 actionset runmodel on multiple threads

acordes — Tue, 03 May 2022 16:27:50 GMT

@BrunoMueller can you help me out?

Re: how to run ds2 actionset runmodel on multiple threads

acordes — Tue, 03 May 2022 22:04:37 GMT

I think that I'm close to the solution.

I run the runmodel.action which executes but slowly.

I copy/paste the log information to the thread program.

It creates the desired table including the prediction variables but quits before writing any observations to it.

The error message is.

ERROR: Line 14: Error reported by DS2 package d2score:
ERROR: Line 14: Store identified by the key 6031C39C5FDB19CC9D86D42C7A7E7FE76987AEB2 is not in memory.

When using the runmodel action the store is found and executes correctly.

cas mysession sessopts=(caslib="casuser");
proc cas; 
   session mysession; 
   source myprogram1;
      thread datagen / overwrite=yes;              
          dcl package score _BAY103MSXS55GR1KJPG052A1W();
          dcl double "P_target_recovery1" having label n'Predicted: target_recovery=1';
          dcl double "P_target_recovery0" having label n'Predicted: target_recovery=0';
          dcl nchar(32) "I_target_recovery" having label n'Into: target_recovery';
          dcl nchar(4) "_WARN_" having label n'Warnings';
          dcl double EM_EVENTPROBABILITY;
          dcl nchar(8) EM_CLASSIFICATION;
          dcl double EM_PROBABILITY;
          varlist allvars [_all_];
                    
          method init();
          _BAY103MSXS55GR1KJPG052A1W.setvars(allvars);
          _BAY103MSXS55GR1KJPG052A1W.setkey(n'6031C39C5FDB19CC9D86D42C7A7E7FE76987AEB2');
          end;
                   method post_BAY103MSXS55GR1KJPG052A1W();
          dcl double _P_;
                       if "P_TARGET_RECOVERY0" = . then "P_TARGET_RECOVERY0" = 0.6131360717;
          if "P_TARGET_RECOVERY1" = . then "P_TARGET_RECOVERY1" = 0.3868639283;
          if MISSING("I_TARGET_RECOVERY") then do ;
          _P_ = 0.0;
          if "P_TARGET_RECOVERY1" > _P_ then do ;
          _P_ = "P_TARGET_RECOVERY1";
          "I_TARGET_RECOVERY" = '1';
          end;
          if "P_TARGET_RECOVERY0" > _P_ then do ;
          _P_ = "P_TARGET_RECOVERY0";
          "I_TARGET_RECOVERY" = '0';
          end;
          end;
          EM_EVENTPROBABILITY = "P_TARGET_RECOVERY1";
          EM_CLASSIFICATION = "I_TARGET_RECOVERY";
          EM_PROBABILITY = MAX("P_TARGET_RECOVERY1", "P_TARGET_RECOVERY0");
                   end;
                 
          method run();
          set "CASUSER(DKXENLO)"."bigdata";
          _BAY103MSXS55GR1KJPG052A1W.scoreRecord();
          post_BAY103MSXS55GR1KJPG052A1W();
end;
         method term();
end;


      endthread;

/* Run the DATA program on all CAS workers. */
   data bigdata_sc / overwrite=yes;                          
         dcl thread datagen dg;                              
         method run();                                       
            set from dg threads=8;                       /*5*/
         end;
      enddata;
   endsource;

   ds2.runDS2 program=myprogram1; 
   run;

Re: how to run ds2 actionset runmodel on multiple threads

BrunoMueller — Thu, 05 May 2022 08:23:35 GMT

I am sorry but I can not help with this.

Re: how to run ds2 actionset runmodel on multiple threads

acordes — Wed, 18 May 2022 14:46:13 GMT

Hi Koen @sbxkoenk

Perhaps you know the solution. I should have placed this post in data science I guess.

Re: how to run ds2 actionset runmodel on multiple threads

sbxkoenk — Wed, 18 May 2022 19:47:11 GMT

Yes, you should have put it in 'SAS Data Science Board' indeed.

Is it possible for you to open a track at Technical Support in your country?

Technical Support (TS) knows about this problem.

I refer you to TS because the root cause and the work-around (there is no real 'solution') is a very long explanation that they can probably do better than me.

BR,

Koen