Issue with multi processing

[mnjtrana]

Hi All,

I am facing peculiar issues with parallel prcessing in sas9.2 batch. My program creates 10 remote sessions and then more than 100 reports are submitted parallely to the open sessions.

However we came across few scenarios where 1 reports is submiited twice to 2 parallel sessions. We are not sure how this is happening, because we have been using this for many years now. 

And this scenario is happening randomly. It happens once in a month or so. We tried everything but didnt find any fix. I am thinking this has to do something with process not completed and being submitted again.

FYI this is not the actual prod code, i tried to create a dummy one, so please ignore any syntax/logic errors.

Appreciate the help in advance!

Thanks

%let all_rep=100;

%macro new;
%do i =1 to &all_rep;
%multipc(&&rep_&i);
%end;
%mend;

%macro multipc(statemnt);,
options autosignon=remote=sess1;
signon sess1 user= fsfsf passwd=;
rsubmit sess1 wait= yes connectpersist=yes cmacvar=stat_sess1
      &statemnt;
endrsubmit;
signoff;
%mend;

%new;

[ballardw]

You have many macro variables we can't see.

Your dummy code has an issue with a starting " on this line:

"%multipc(&&rep_&i);

without a match.

And the comma on this line is suspect as well:

%macro multipc(statemnt);,

You also only have one %mend statement but are defining two macros in the shown code.

It is almost never a good idea to define a macro in another macro. Calling a defined macro multiple times is fine but redefining it multiple times is a waste of CPU cycles at a minimum and potentially a horror waiting to be debugged.

I would suggest that you generate the values of the &&rep_&i with simple %put statements. Not executing any other code but what may be needed to get the values of &&rep_&i.The %put should include enough information to indicate which iteration of the loop is currently executing and possibly what the &rep_ value may be.

I think that you might find that you are creating the same value more than once. But you have at least 100 macro variables we cannot see in any way on the forum.

[mnjtrana]

Hey, thanks for the reply. As i mentioned, this is not the actual prod code, i tried to create a dummy one, so please ignore any syntax/logic errors.
However we came across few scenarios where 1 reports is submiited twice to 2 parallel sessions. We are not sure how this is happening, because we have been using this logic without any issues for many years now.

And this scenario is happening randomly. It happens once in a month or so. We tried everything but didnt find any fix. I am thinking this has to do something with process not completed and being submitted again.

[ballardw]

---

@mnjtrana wrote:
Hey, thanks for the reply. As i mentioned, this is not the actual prod code, i tried to create a dummy one, so please ignore any syntax/logic errors.
However we came across few scenarios where 1 reports is submiited twice to 2 parallel sessions. We are not sure how this is happening, because we have been using this logic without any issues for many years now.

And this scenario is happening randomly. It happens once in a month or so. We tried everything but didnt find any fix. I am thinking this has to do something with process not completed and being submitted again.

---

Doesn't  invalidate the suggestion to generate all the values of parameters you may be passing to the process. The indirect referencing using &&rep_&i might generate the same result. Especially if &rep changes. You extracted code does not show where &rep might be getting values. So that might be the place to look.

[Kurt_Bremser]

This is just a disaster waiting to happen. What if one of the macro variables contains a comma? Since you obviously have SAS code in there, that's very likely to happen (SQL needs LOTS of commas).

And wait=yes in the rsubmit statement defeats the purpose, as all code will be run sequentially anyway, only with creating lots of sessions with lots of overhead. And since you sign on and sign off in the inner macro, you don't use 10 sessions, but 100.

If you want to run jobs in parallel, let the scheduler handle it, as it is designed for this purpose. Or do the parallelization from a shell script. 

[Amir]

Hi,

Separate to the pseudo code you've supplied, are you able to share any logs showing where the same report is requested more than once?

Kind regards,

Amir.