topic Re: Query Wikipedia API using PROC HTTP in SAS Procedures

Query Wikipedia API using PROC HTTP

Criptic — Fri, 30 Jun 2017 08:27:08 GMT

Hey 🙂

I'm trying to query the Wikipedia-API and I get the following Error: SSL Error: Missing CA trust list.

I tried with adding perm file via the options (options set= SSLCALISTLOC="/path_to_file/cacert-2017-06-07.pem"; I got the pem-file from here: https://curl.haxx.se/docs/caextract.html

My code looks like this:

options set= SSLCALISTLOC="path_to_file/cacert-2017-06-07.pem";
filename resp 'path_to_file/aaa.txt';

proc http
	url="https://en.wikipedia.org/w/api.php?action=query&titles=Main%20Page&prop=revisions&rvprop=content&format=json"
	out=resp
	method='GET'
	proxyhost="10.120.11.11"
	proxyport=8080;
run;

If I'm using a HTTP site everything works like a charm, the error only occurs with https.

I'm currently on SAS 9.4 M2 - can somebody please give me pointers.

Thanks in advance

David

Re: Query Wikipedia API using PROC HTTP

ChrisHemedinger — Fri, 30 Jun 2017 12:51:35 GMT

Sounds like a cool project! I've been meaning to look into something similar with Wikipedia. Thanks for the nudge.

Looks like you might not have everything quite right with the TLS requirements. Can you try with a simple "echo" service from httpbin.org? That's a great way to test that your program "plumbing" is working before complicating things with the actual site you want data from.

%macro echoResp(fn=);
    %global _respFilesize;
	data _null_;
		infile &fn;
		input;
		put _infile_;
	run;
%mend;

filename out temp;
filename input TEMP;
filename headout temp;

data _null_;
   file input;
   put "this is some sample text";
run;

proc http 
   url="https://httpbin.org/post"
   in=input
   out=out
   headerout=headout;
run;

%put Headers Out:;
%echoResp(fn=headout);

%put Response:;
%echoResp(fn=out);

My log:

Headers Out:
59         %echoResp(fn=headout);

NOTE: The infile HEADOUT is:
      Filename=#LN00106,
      RECFM=V,LRECL=32767,File Size (bytes)=302,
      Last Modified=30Jun2017:08:47:16,
      Create Time=30Jun2017:08:47:16

HTTP/1.1 200 OK
Connection: keep-alive
Server: meinheld/0.6.1
Date: Fri, 30 Jun 2017 12:47:17 GMT
Content-Type: application/json
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
X-Powered-By: Flask
X-Processed-Time: 0.00111889839172
Content-Length: 404
Via: 1.1 vegur
NOTE: 12 records were read from the infile HEADOUT.
      The minimum record length was 0.
      The maximum record length was 38.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds
      

60         
61         %put Response:;
Response:
62         %echoResp(fn=out);

NOTE: The infile OUT is:
      Filename=#LN00104,
      RECFM=V,LRECL=32767,File Size (bytes)=404,
      Last Modified=30Jun2017:08:47:16,
      Create Time=30Jun2017:08:47:16

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {
    "this is some sample text\r\n": ""
  }, 
  "headers": {
    "Accept": "*/*", 
    "Connection": "close", 
    "Content-Length": "26", 
    "Content-Type": "application/x-www-form-urlencoded", 
    "Host": "httpbin.org", 
    "User-Agent": "SAS/9"
  }, 
  "json": null, 
  "origin": "104.x.x.x", 
  "url": "https://httpbin.org/post"
}
NOTE: 19 records were read from the infile OUT.
      The minimum record length was 1.
      The maximum record length was 57.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds

Re: Query Wikipedia API using PROC HTTP

Criptic — Mon, 03 Jul 2017 05:48:24 GMT

Thank you for your reply. I tried your suggestions but I still get the same error: SSL Error: Missing CA trust list. Do you have any further suggestions or ideas?

Re: Query Wikipedia API using PROC HTTP

ChrisHemedinger — Mon, 03 Jul 2017 12:00:25 GMT

Hi @Criptic,

Try changing your syntax to remove the SET= part:

options SSLCALISTLOC="path_to_file/cacert-2017-06-07.pem";

If your SAS session makes use of other SSL interactions, making this change could affect those other operations. The list of certs and behavior depends on whether you are using SAS 9.4 m1 or m2, or m3/m4. See the doc for this option.

There is another option, SSLCACERTDIR Environment Variable, that allows you to "inject" a cert path into your session. See that topic for a full description of the precendence list for SSL certs.

Re: Query Wikipedia API using PROC HTTP

Criptic — Mon, 03 Jul 2017 12:30:43 GMT

This documentation was exactly what I needed, thank you!