We've build a log scanner which make use of de viya job api through the following code:
%let base_uri=%sysfunc(getoption(SERVICESBASEURL));
proc http method=get oauth_bearer = sas_services
url="&base_uri/jobExecution/jobs/?limit=3000"
out=jobrec;
run;
Starting from the end of September, it sometimes gives an error. The cause of this error is the fact that the api give a html status 500 internal server.
I have replicated using api calls from curl.
curl -o results https://xxx.ondemand.sas.com/jobExecution/jobs/?limit=3000 --cacert xxxx -H "Authorization: Bearer $BEARER_TOKEN"
if the result is smaller (limit set to <1000) I can get results almost immediately and for each call, for bigger results we are getting htt500.
I run the same in the loop and around 10% calls were successfull;
The output in size is around 60mb (slightly less) with limits starting from 1000.
The maximum size with limit 3000 (used by customer) is 59.8mb
-rw-r--r-- 1 splbrk UsersGrp 59629324 Nov 2 04:31 results (limit 1000)
-rw-r--r-- 1 splbrk UsersGrp 60200103 Nov 2 04:31 results2 (limit 1200)
-rw-r--r-- 1 splbrk UsersGrp 62686500 Nov 2 04:41 results3 (limit 3000)
I would say that ~60mb is something that doesnt make happy our calls even this is still very small output.
Is there any limit? Any options to steer? I have found in tessa similar issue but for Viya 3.5
Best Regards,
Bartosz
