Using PROC HTTP 2 - Accessing Internet Information and Debugging
Introduction
In a previous post ( Using PROC HTTP 1: Reviewing the HTTP Protocol) we discussed the basics of the HTTP protocol, and experimented using several cURL (client for URL) examples to obtain data from an API via the internet. While cURL is the de facto standard for working with APIs from the command line, we’re SAS programmers – and, of course, we wantSASto do the work for us! In the Using FILENAME URL to Access Internet Information post, we experimented with FILENAME URL, but noted that PROC HTTPwas significantly more flexible. Let’s explore replicating the cURL code and FILENAME URL examples using PROC HTTP and debugging unexpected results.
The Basics
URIs for API Access
Remember that API resources are accessed via a URI, and a URI is made up of several parts:
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
Now, you’re probably used to calling that a URL - and so does PROC HTTP. The URI can be specified using theURLparameter. The general PROC HTTP syntax is:
PROC HTTP
URL="target-URI"
< METHOD ="<http-method>"
< authentication-type-options >
<header-options>
<web-server-authentication-options>
<proxy-server-connection-options>
<other-options>
;
< DEBUG options;>
< HEADERS "HeaderName"="HeaderValue" < "HeaderName-n"="HeaderValue-n" >;>
Read the API documentation! While the httpbin.org examples shown here work, the syntax for each API is unique. The documentation will show how to write a URI to get your desired resource, and how to handle authentication.
Simple GET request
cURL code:
curl -X GET http://httpbin.org/get
SAS code:
/* Simple GET request */
filename resp temp;
proc http
url="https://httpbin.org/get"
method="GET"
out=resp;
run;
/* Review JSON response in the log */
data _null_;
rc = jsonpp('resp', 'LOG');
run;
JSON results:
GET request with custom headers
cURL code:
curl -H "Accept:application/xml \" -H "Content-Type:application/json"\ -H "User-Agent:MySASCode" httpbin.org/xml
| Header Text | Description |
| Accept:application/xml | Describes the type of data the requesting application expects in the return, in this case, XML. |
| Content-Type:application/json | Describes the format of the request being sent, in this case, JSON. |
| User-Agent:MySASCode | Overrides the default description of the browser or agent being used to send the request. |
SAS code:
/* GET request with custom headers */
filename resp temp;
proc http
url="https://httpbin.org/xml"
method="GET"
out=resp;
headers "Accept"="application/xml"
"Content-Type"="application/json"
"User-agent"="MySASCode";
run;
/* Review XML response in the log */
data _null_;
infile resp;
input;
put _infile_;
run;
XML Response:
POST request
cURL code:
curl -X POST "https://httpbin.org/post" \
-H "accept: application/json" \
-d "My posted text."
SAS code:
/*POST request */
filename input temp;
filename resp temp;
data _null_;
file input;
put "My posted text.";
run;
proc http
url="https://httpbin.org/post"
method="POST"
in=input
out=resp;
run;
JSON Response:
PUT request
cURL code:
curl -X PUT "https://httpbin.org/put" \
-H "accept: application/json" \
-d "My text update."
/*PUT request */
filename input temp;
filename resp temp;
data _null_;
file input;
put "My text update.";
run;
proc http
url="https://httpbin.org/put"
method="PUT"
in=input
out=resp;
run;
JSON Response:
DELETE request
cURL code:
curl -X DELETE "https://httpbin.org/delete?file=badfile.html" \
-H "accept: application/json"
SAS code:
/* DELETE request */
filename resp temp;
proc http
url="https://httpbin.org/delete?file=badfile.html"
method="DELETE"
out=resp;
headers "Accept"="application/json";
run;
JSON Response:
Authentication with PROC HTTP
HTTP Basic - username and password
cURL code:
curl -H "Accept:application/xml" \
-H "Content-Type:application/json" \
-H "User-Agent:MyCustomAgent" httpbin.org/xml
SAS code:
/* HTTP Basic - username and password */
filename resp temp;
proc http
url="https://httpbin.org/basic-auth/MyID/Pa55w0rd"
method="GET"
auth_basic
WebUsername="MyID"
WebPassword="Pa55w0rd"
out=resp;
run;
JSON Response:
Bearer Token
cURL code:
curl -X GET "https://httpbin.org/bearer" \
-H "accept: application/json" \
-H "Authorization: Bearer gakdfdadfkae213913"
SAS code:
/* Bearer token*/
filename resp temp;
proc http
url="https://httpbin.org/bearer"
method="GET"
oauth_bearer="gakdfdadfkae213913"
out=resp;
run;
/* Review response in the log */
data _null_;
infile resp;
input;
put _infile_;
run;
filename resp clear;
JSON Response:
Proxy Server Authentication
cURL code:
curl -x http://proxy.example.com:8080 \
-U your_proxy_username:your_proxy_password \
http://example.com/api/data
SAS code:
/* Proxy Server Authentication */
/* NOTE: The syntax is valid, but this code does not
actually run, so no response is shown. */
filename resp temp;
proc http
url="http://example.com/api/data"
method="GET"
out=resp
proxyhost="proxy.example.com"
proxyport=8080
proxyusername="myProxyID"
proxypassword="myProxyPassword";
run;
Converting FILENAME URL Code
Capturing Response Headers
FILENAME URL code:
/* Capturing the response headers */
filename head temp;
filename in url "https://httpbin.org/xml" headers=head;
/* After read from the URL, the headers have been captured */
data _null_;
infile head;
input;
put _infile_;
run;
filename head clear;
filename in clear;
JSON Response:
PROC HTTP Code:
/* Capturing Response Headers */
filename resp temp;
filename hdrout temp;
proc http
url="http://httpbin.org/xml"
method="GET"
out=resp
headerout=hdrout;
run;
/* Review response headers in the log */
data _null_;
infile hdrout;
input;
put _infile_;
run;
filename resp clear;
filename hdrout clear;
JSON Response:
Note the difference in theConnectionsection of the headers:
- FILENAME URL: the connection is closed after use, freeing up resources, but introducing latency.
- PROC HTTP: uses a keep-alive to reduce latency by avoiding the overhead of establishing a new connection for each subsequent request.
Real-World Application: World Bank API
Converted from FILENAME URL code:
/* NOTE: Please run create_work.countries.sas before this code to create work.countries */
/* The FILENAME URL statement was:*/
/*filename in url 'https://api.worldbank.org/v2/country/all/indicator/SP.POP.TOTL?date=2020&format=json&per_page=5000';*/
/* Replaced the FILENAME statement with this code: */
filename in temp;
proc http
url='https://api.worldbank.org/v2/country/all/indicator/SP.POP.TOTL?date=2020&format=json&per_page=5000'
method="GET"
out=in
;
run;
/* The remaining report code remains the same */
filename jmap temp;
libname in json map=jmap automap=replace ;
/* Join the the World Bank data to the SAS table to create the Country_pop_2020 data set */
proc sql;
create table Country_pop_2020 as
select Country
,code
,value format=comma15. as Population
from in.root as r
inner join
countries as c
on r.countryiso3code=c.Code
order by value desc
;
quit;
/* Don't need the World Bank connection anymore. Clear librefs and filerefs */
libname in clear;
filename in clear;
filename jmap clear;
/* Create the report based on our new table */
title "Countries with Populations >100,000,000";
ods layout gridded width=6.5in columns=2;
ods region;
proc sql;
select *
from Country_pop_2020
where Population >100000000
order by Population desc
;
quit;
ods region;
proc sgplot data=country_pop_2020;
vbar Country / response=Population categoryorder = respdesc;
where Population >100000000;
run;
ods layout end;
title;
Results:
Debugging
The PROC HTTP DEBUG statement can produce additional information useful for troubleshooting. I generally use LEVEL= to set the debug level:
| LEVEL= value | Description |
| 1 | Displays request and response headers in the log. |
| 2 | Displays level 1 info + request body |
| 3 | Displays level 2 info + response body |
Here are some other useful options:
| Debug Option | Description |
| NO_REQUEST_HEADERS | Suppresses the request header |
| NO_RESPONSE_HEADERS | Suppresses the response header (for LEVEL >=2) |
| NO_REQUEST_BODY | Suppresses the request body (for LEVEL >=2) |
| NO_RESPONSE_BODY | Suppresses the response body (for LEVEL =3) |
| OUTPUT_TEXT |
Displays the request and response bodies as text NOTE: Always use this when LEVEL=3 is specified. |
Scenario 1:
SAS code:
/* A GET request for a non-existing resource */
filename resp temp;
proc http
url="https://httpbin.org/bad_file_name.html"
method="GET"
out=resp;
run;
/* Review JSON response in the log */
data _null_;
rc = jsonpp('resp', 'LOG');
run;
From the Log
> GET /bad_file_name.html HTTP/1.1
> User-Agent: SAS/9
> Host: httpbin.org
> Accept: */*
> Connection: Keep-Alive
>;
< HTTP/1.1 404 NOT FOUND
<Date: Wed, 09 Apr 2025 18:05:26 GMT
<Content-Type: text/html
<Content-Length: 233
<Connection: keep-alive
<Server: gunicorn/19.9.0
<Access-Control-Allow-Origin: *
<Access-Control-Allow-Credentials: true
<
NOTE: 404 Not Found
Scenario 2:
The SAS code in this scenario initially produced a NOTE: 401 Unauthorized message in the log when first executed. I added DEBUG LEVEL=3 to get more details about why the operation didn’t work. Because I was using DEBUG LEVEL=3, I also specified OUTPUT_TEXT to forestall system instability if the response data was binary. This option directs PROC HTTP to handle both request and response body data as text.
SAS code:
filename input temp;
filename resp temp;
data _null_;
file input;
put "*** My text update ***";
run;
proc http
url="https://httpbin.org/post"
method="PUT"
in=input
out=resp;
debug level=3 output_text;
run;
filename input clear;
filename resp clear;
From the Log:
> PUT /post HTTP/1.1
> User-Agent: SAS/9
> Host: httpbin.org
> Accept: */*
> Connection: Keep-Alive
> Content-Length: 24
> Content-Type: application/octet-stream
>
> *** My text update ***
< HTTP/1.1 405 METHOD NOT ALLOWED
< Date: Wed, 09 Apr 2025 15:05:45 GMT
< Content-Type: text/html
< Content-Length: 178
< Connection: keep-alive
< Server: gunicorn/19.9.0
< Allow: OPTIONS, POST
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
<
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>405 Method Not Allowed</title>
<h1>Method Not Allowed</h1>
<p>The method is not allowed for the requested URL.</p>
NOTE: 405 Method Not Allowed
In the log, I see that I’ve used the wrong URI for my PUT request.
Wrap-up
Holy mackerel! That was a lot of information! PROC HTTP sure is a powerful tool for accessing and acquiring data from the internet, and for working with APIs. You stuck with it this far - so what do you think? What applications might you have for PROC HTTP? I hope you found this information useful.
Mark
On a personal note: I'm retiring from SAS as of April 30, 2025, so, while this is my last blog post as a SAS employee, it won't be my last post - I'll be back! I will be at IASUG, NEBSUG, and MSUG next month. I'm also co-chairing the Hands-On Workshops (HOW) section for SESUG this year (the SESUG 2025 Call for Abstracts currently open - submit a proposal here) Will I see you in September?
Of course, I'll continue to poke around here on SAS Communities - that's a hard habit to break!
PS: Download a ZIP file containing PDF of posts in this series, and the cURL and SAS code. Get it here:https://bit.ly/SASJediPROCHTTP
Other resources:
Related posts:
Find more articles from SAS Global Enablement and Learning here .
