- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content
I have this code:
/* Mock dataset */
data DaCensire;
do i=1 to 4000;
id=cats('z',put(rand('integer',1,999999),z6.));
Newgroup='XXX';
output;
end;
run;
data DaCensire;
set DaCensire;
call symputx('NCensimenti',_N_);
call symputx(cats('censimento',_N_), id);
call symputx(cats('gruppo',_N_), Newgroup);
run;
%macro assegnazioneGruppo;
%do h=1 %to &NCensimenti;
%put Inserisco utenza &h/&NCensimenti e &&censimento&h nel gruppo &&gruppo&h;
proc http url="https://xxx/identities/groups/&&gruppo&h/userMembers/&&censimento&h"
method="PUT"
OAUTH_BEARER=SAS_SERVICES;
debug level = 1;
run;
%put Fine inserimento utenza &h/&NCensimenti e &&censimento&h nel gruppo &&gruppo&h;
%end;
%mend;
%assegnazioneGruppo;
The macrovar NCensimenti is about 4.000.
This code works exactly fine, but a really weird thing happens: the 26th iteration runs to the end (I see in the log the last %put), but the 27th iteration never starts (I don't see the first %put) and the code freezes forever.
I taught there may be something with the 27th value, but when I run the whole code again the list is randomized, so the 27th value changes every time but still, exactly at the 27th iteration, it never starts.
What may be happening?
Thanks
Regards
- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content
Can you provide some logic to generate dummy data that represents the data on 'DaCensire'? That would be helpful in understanding the logic better. Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content
When something regarding a web site or service quits working at a fixed number of iterations regardless of the data content I start to suspect some sort of setting at the web site. You might be triggering a defense against denial of service attacks, web crawlers or any number of things hitting the same site numerous times in a short period of time.
You did not attach any LOG to inspect.
- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content
Because there is nothing to inspect in the log, it just freezes... The web site is the standard SASViya REST APIs for handling identities.
I add here the log from the 25th to the end of the log. I've added some red marks (*) to comment that line after:
....... Inserisco utenza 25/2083 e z123456 nel gruppo XXX (*1) > PUT /identities/groups/XXX/userMembers/z123456 HTTP/1.1 > User-Agent: SAS/9 > Host: sasviya > Accept: */* > Authorization: Bearer > Connection: Keep-Alive > Content-Length: 0 > Content-Type: application/octet-stream > < HTTP/1.1 201 < Cache-Control: no-cache, no-store, max-age=0, must-revalidate < Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; img-src 'self' *.sas.com blob: data:; style-src 'self' 'unsafe-inline'; child-src 'self' blob: data: mailto:; < Date: Tue, 25 Jun 2019 11:35:23 GMT < Expires: 0 < Pragma: no-cache < Server: Apache/2.4 < Strict-Transport-Security: max-age=31536000 ; includeSubDomains < Vary: User-Agent < X-Content-Type-Options: nosniff < X-Frame-Options: SAMEORIGIN < X-XSS-Protection: 1; mode=block < Content-Length: 0 < Connection: keep-alive < NOTE: PROCEDURE HTTP ha utilizzato (tempo totale di elaborazione): (*2) real time 0.21 seconds cpu time 0.02 seconds Fine inserimento utenza 25/2083 e z123456 nel gruppo XXX (*3) Inserisco utenza 26/2083 e z123457 nel gruppo XXX (*4) > PUT /identities/groups/XXX/userMembers/z123457 HTTP/1.1 > User-Agent: SAS/9 > Host: sasviya > Accept: */* > Authorization: Bearer > Connection: Keep-Alive > Content-Length: 0 > Content-Type: application/octet-stream > < HTTP/1.1 201 < Cache-Control: no-cache, no-store, max-age=0, must-revalidate < Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; img-src 'self' *.sas.com blob: data:; style-src 'self' 'unsafe-inline'; child-src 'self' blob: data: mailto:; < Date: Tue, 25 Jun 2019 11:35:33 GMT < Expires: 0 < Pragma: no-cache < Server: Apache/2.4 < Strict-Transport-Security: max-age=31536000 ; includeSubDomains < Vary: User-Agent < X-Content-Type-Options: nosniff < X-Frame-Options: SAMEORIGIN < X-XSS-Protection: 1; mode=block < Content-Length: 0 < Connection: keep-alive < NOTE: PROCEDURE HTTP ha utilizzato (tempo totale di elaborazione): (*5) real time 0.04 seconds cpu time 0.00 seconds Fine inserimento utenza 26/2083 e z123457 nel gruppo XXX (*6) (*7)
Comments:
- It corresponds to the very first %put of the cycle, so the 24th iteration has just started
- The proc http called with the 24th user ended and is successful
- It corresponds to the very last %put of the cycle, so the 24th iteration has just completely ended
- It corresponds to the very first %put of the cycle, so the 25th iteration has just started
- The proc http called with the 25th user ended and is successful
- It corresponds to the very last %put of the cycle, so the 25th iteration has just completely ended
- !!! It doesn't even start the next cycle with the first %put! If the problem was the proc http, I would expect that at least the first %put of the 27th iteration would have been printed! But it just stops doing anything, the code remains running forever, and nothing else happens.
- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content
What interface are you using to submit the SAS code?
- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content
At first with Sas Studio V. Then directly with a scheduled Job (so without an interface, like command line) redirecting the log to a file.
In both the cases the behavior is the same as I described above.
Thank you
Regards
- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content
Ok, the best next step I can think of is to try to get some additional logging, from when it hangs.
When you submit from the command line, I want you to specify the following command line option:
-LOGPARM "WRITE=IMMEDIATE"
In your SAS program, let's also increase the log verbosity by adding the following line of code at the top of the program
OPTIONS MLOGIC MLOGICNEST MPRINT MPRINTNEST MSGLEVEL=I;
In your HTTP Procedure call you use the debug statement. Let's also increase the level
PROC HTTP [...];
DEBUG LEVEL = 3;
RUN;
This all will produce a significantly larger log file, but specifically, what we care about most is if any additional log information is produced when the loop gets 'stuck' and the process appears to hang. As well as any potential logging produced when you, I assume, terminate the process.
- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content
Thanks!
I'm having difficulties setting
-LOGPARM "WRITE=IMMEDIATE"
because the program is launched as a job in the SASJobExecution (so it's like a batch but there isn't a real command line where I can put parameters) and inside I redirect the log to a file with proc printto. How to set the LOGPARM using SASJobExecution?