proc http cannot return large GZIP Files

[daRocha]

Hi all

My question is related with the issue reported here:

https://communities.sas.com/t5/Developers/proc-http-can-not-return-GZIP-file-gt-64-KB/td-p/645154

I am facing the exact same issue and in the original thread, even though the issue is marked as solved, the last comment made by @SimonDawson  says:
"This will be looked at and addressed in a future release."

Has any progress been done or is there a workaround?

Thanks,

daRocha

[andreas_lds]

Are you using the latest sas release with hotfixes applied?

[daRocha]

All updates are in place. I am currently running my code on SAS 9.4. In particular: 9.04.01M7P080520

[andreas_lds]

Open a track with sas tech support, they should know if the bug will be fixed with m8 or if the plans to fix the issue have been abandoned.

[joeFurbee]

Hi @daRocha,

I checked with SAS R&D and found out:

The fix is in 9.4m7 and beyond and in Viya.

Thanks,

Joe

[daRocha]

Hi @joeFurbee 

With the tests I ran it does not seem to be working with my version (9.04.01M7P080520) 

For example, this code works for small files:

/*API Endpoint */
%let urlLink=https://reference-data-api.kaiko.io/v1/exchanges;

/*Temp work location*/
%let dataLoc=%sysfunc(getoption(work))\kkExhange;
/*Create location and assign lib*/
options dlcreatedir;
libname kkExchan "&dataLoc.";
options nodlcreatedir;

/* Results file*/
filename KaikoEx "&dataLoc.\metaKaiko.json.gz";

proc http 
  url="&urlLink."
    method='get'
    out=KaikoEx;
  headers
    'Accept' = 'application/json'
    'Accept-Encoding'= 'gzip, deflate, br';
run;

/*Filename poiting to the results file as gzip*/
filename KaikoEx zip "&dataLoc.\metaKaiko.json.gz" gzip;
filename KaikMap temp;

/*Json libname */
libname KaikoExc json fileref=KaikoEx map=KaikMap automap=create;

/*List json file contents*/
proc contents data=kaikoExc._all_;
run;

but when I use the same code for a larger gzip file it does not work. For large files, I am using the API Endpoint to test:
https://reference-data-api.kaiko.io/v1/instruments 

Is there something wrong with my code or do I need to update my SAS version?

Thank you,

daRocha 

[joeFurbee]

Hi @daRocha,

Your SAS version should have the patch. We may have to open a tech support ticket to investigate. Before we do that, the developer I spoke with asked the following:

Can he provide the debug logs for the run? Setting logger App.tk.HTTPC to debug would be good to see what is going on.

Can you provide the logs for us here?

[joeFurbee · Accepted solution]

Hi @daRocha,

The developer took another look at this and while the error seems to be suppressed in his testing the file is not processed. IOW, we still have an issue. I'll follow the progress of the fix and will update you when it's fixed and in which version of SAS 9.

Thanks,

Joe