BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
KTanner
Fluorite | Level 6

Hello everyone,

 

I am having some trouble sending a POST request to a REST API that requires content-type 'multipart/form-data' and some parameters that point to a path containing a JSON file and a type. The cURL equivalent (that works today using Base SAS and an X command calling a shell script) looks like this:

 

curl -X POST -H 'X-API-TOKEN: tokenwenthere' -F 'contacts=@\\path\to\upload\Contact_Upload.json;type=application/json' 'https://co1.qualtrics.com/API/v3/mailinglists/ML_mailinglistid/contactimports' -o '\\path\to\response\response.txt'

However, I want to move this process to Enterprise Guide, which in my instance is connected to a SAS server where cURL doesn't work. With SAS 9.4M3's enhancements to PROC HTTP, I should be able to do this within the procedure. But I am having trouble getting the form data that the call needs as a parameter to work with the procedure. Below is the code that I am running:

 

filename resp2 '\\path\to\apiresponse\response.txt';
filename test '\\path\to\headerout\test.txt';

proc http method="POST"
	url="&link"
	in='contacts=@\\Path\to\upload\Contact_Upload.json;type=application/json'
	ct="multipart/form-data"
	headerout=test
	out=resp2;
	headers
		"X-API-TOKEN"="tokenwenthere";

run;

When I run this code, I get nothing from the API as a response, but the headerout does give me some information:

 

HTTP/1.1 413 Request Entity Too Large
Request-Time: 1
Content-Length: 0
Date: Tue, 17 Jan 2017 02:20:20 GMT
Connection: close

 

On the API documentation, it mentions that a 413 error could be a result of a malformed multipart/form-data request. Which is why I am suspecting that there is a problem within the HTTP procedure with how it handles multipart/form-data requests.

 

Has anyone run in to this problem and can help?

1 ACCEPTED SOLUTION

Accepted Solutions
JosephHenry
SAS Employee

Editor's note: See the final working code in this message later in the thread.

 

Like Chris said, unfortunately proc http does not yet natively support sending multipart data.

 

You can pretty easily use the datastep to format a document that follows the mutipart format.

 

For your example you would do somthing like this:

 

filename resp2 '\\path\to\apiresponse\response.txt';
filename test '\\path\to\headerout\test.txt';
filename in TEMP;
/* Set the boundary identifier. It does not matter what this is, as long as its unique */
%let boundary=thisisauniqueid; 

data _null_;
infile test end=eof;
file in;

/* for each file we are sending, we need to add some special headers at the beginning*/
if _n_ = 1 then
 do;
  put "--&boundary."; /* This separates each data piece as a separate entity. Must start with -- */
  put 'Content-Disposition: form-data; name="file1"; filename="file1.txt"'; /* This is the content identifier */
  put "Content-Type: text/plain"; /* This is the acutaly content type of the entity in the multi-part form*/
  put ; /* Must end with a CRLF signaling that what comes next is the actual entity */
end;
input;
put _infile_; /* add the actual file to be sent*/
/* the end of the multipart blob needs to be terminated */
if eof then
 do;
   put ; /* Must have a CRLF*/
   put "--&boundary--"; /* must start and end with --*/
 end;
run;

/* now that the file is created, we can send it to our server*/
proc http url="&link."
 in=in
 ct="multipart/form-data; boundary=&boundary."
 out=out;
 run;

If you are sending multiple files, the code is pretty similar, but you would need to do multiple datasteps.

 

The first datastep you would be identical to what is above, except you would leave off the EOF bit.

 

for Subsequent files, you would need to add the "mod" option for your file and also add a CRLF before the boundary. so just do a put;

 

The file would then just need to be terminated like it was ealier. This would be pretty easy to wrap into a few macros. If you need a full example of sending multiple files, just let me know.

 

Hope this helps.

View solution in original post

8 REPLIES 8
ChrisHemedinger
Community Manager

According to the PROC HTTP developer, the procedure does not (yet) support multipart form data.   You can achieve what you want (I think) by using DATA step to create the multipart form as a file, and then use the IN= parameter to point to that fileref.

 

I'm sorry I don't have an example.  I've asked the developer if he can come up with one -- if that happens, we'll post back.

SAS Hackathon registration is open! Build your skills. Make connections. Enjoy creative freedom. Maybe change the world.
JosephHenry
SAS Employee

Editor's note: See the final working code in this message later in the thread.

 

Like Chris said, unfortunately proc http does not yet natively support sending multipart data.

 

You can pretty easily use the datastep to format a document that follows the mutipart format.

 

For your example you would do somthing like this:

 

filename resp2 '\\path\to\apiresponse\response.txt';
filename test '\\path\to\headerout\test.txt';
filename in TEMP;
/* Set the boundary identifier. It does not matter what this is, as long as its unique */
%let boundary=thisisauniqueid; 

data _null_;
infile test end=eof;
file in;

/* for each file we are sending, we need to add some special headers at the beginning*/
if _n_ = 1 then
 do;
  put "--&boundary."; /* This separates each data piece as a separate entity. Must start with -- */
  put 'Content-Disposition: form-data; name="file1"; filename="file1.txt"'; /* This is the content identifier */
  put "Content-Type: text/plain"; /* This is the acutaly content type of the entity in the multi-part form*/
  put ; /* Must end with a CRLF signaling that what comes next is the actual entity */
end;
input;
put _infile_; /* add the actual file to be sent*/
/* the end of the multipart blob needs to be terminated */
if eof then
 do;
   put ; /* Must have a CRLF*/
   put "--&boundary--"; /* must start and end with --*/
 end;
run;

/* now that the file is created, we can send it to our server*/
proc http url="&link."
 in=in
 ct="multipart/form-data; boundary=&boundary."
 out=out;
 run;

If you are sending multiple files, the code is pretty similar, but you would need to do multiple datasteps.

 

The first datastep you would be identical to what is above, except you would leave off the EOF bit.

 

for Subsequent files, you would need to add the "mod" option for your file and also add a CRLF before the boundary. so just do a put;

 

The file would then just need to be terminated like it was ealier. This would be pretty easy to wrap into a few macros. If you need a full example of sending multiple files, just let me know.

 

Hope this helps.

ChrisHemedinger
Community Manager

Thanks to @JosephHenry for the response!  I'll add one tip: you can get SAS to create a unique ID for you:

 

%let boundary=%sysfunc(uuidgen());
SAS Hackathon registration is open! Build your skills. Make connections. Enjoy creative freedom. Maybe change the world.
KTanner
Fluorite | Level 6

Thanks so much for your help! It was very useful, and I am glad to know that form data submission is not natively supported.

 

Unfortunately, the solution provided did not work. I am getting the same 413 error that I had gotten previously. The form data that I need to submit to the API server is:

"contacts=\\path\to\contacts.contacts.json; type=application\json", not the file that contains the upload information. So this is the form data that the API requires as parameters.

 

Is there a way to use a similar process to be able to format these two parameters in the in= statement so that proc http sends them over in a multipart/form-data format that the API can recognize?

 

Thanks again for your help!

ChrisHemedinger
Community Manager

You're using this API, right?

 

It's new to me, but my guess is that the cURL example uses the -F with the @ notation so you can reference a local file with your contacts (JSON format).  When you run cURL, the tool reads the content of that local file and packages it up for transport to the API.  That is, the API server won't be able to see/read your local JSON file directly, right?  

 

With the DATA step approach shown by @JosephHenry, you're basically doing that step that cURL would have done for you.  You would use DATA step to read the local JSON file, and echo its contents and sandwich it between the content headers/metadata, then use that as the IN= on your HTTP POST.  Don't forget the METHOD="POST" (in your original code, but wasn't in Joseph's example).

 

 

SAS Hackathon registration is open! Build your skills. Make connections. Enjoy creative freedom. Maybe change the world.
JosephHenry
SAS Employee
If you run the cURL command with the option --trace and copy me the output, I could modify the example to match.


KTanner
Fluorite | Level 6

Happily, that will not be necessary! I was able to get it to work with a couple of small modifications. I will share them below:

 

@ChrisHemedinger's response about what the data _null_ step was doing in relation to cURL was the big aha moment. At that point, we needed to translate both the path parameter and the type parameter that was being passed in to the form to something that worked within the data _null_ framework that you presented. So, the first thing I did was took Chris's suggestion and incorporated that into the boundary macro variable:

/*Set up the form parameters*/

%let boundary=%sysfunc(uuidgen());

Then, the next important part was to make sure that the name passed through the block of metadata matched the name of the parameter that the API was looking for and that there was a matching file name. Since the data _null_ was doing the unpacking for me, I just needed to make sure that the data had the same parameter name that the API was looking for. 

data _null_;
infile Contact /*This represents the json file containing the contacts and was defined in an earlier fileref*/ 
end=eof;
file in;

if _n_ = 1 then do;
	put "--&boundary.";
	put 'Content-Disposition: form-data; name="contacts"; filename="contact_upload.json"';
	put 'Content-Type: application/json';
	put ;
end;
input;
put _infile_;
put ;
if eof then do;
	put ;
	put "--&boundary.--";
end;
run;

 

And then I needed to specify the type. You had already put the header that specified the content type in the program, it just needed to be changed to a different type.

 

With that it works! Thank you both so much, this will be a huge time saver! For those interested, below is the full code:

 

filename resp2 '\\path\to\APIoutput\response.txt';
filename in TEMP;

/*Set up the form parameters*/

%let boundary=%sysfunc(uuidgen());

data _null_;
infile Contact /*This represents the json file containing the upload data and was defined in an earlier fileref*/ 
end=eof;
file in;

if _n_ = 1 then do;
	put "--&boundary.";
	put 'Content-Disposition: form-data; name="contacts"; filename="contact_upload.json"';
	put 'Content-Type: application/json';
	put ;
end;
input;
put _infile_;
put ;
if eof then do;
	put ;
	put "--&boundary.--";
end;
run;

proc http method="POST"
	url="&link"
	in=in
	ct="multipart/form-data;boundary=&boundary."
	out=resp2;
	headers
		"X-API-TOKEN"="tokenwenthere";

run;

 

ChrisHemedinger
Community Manager

Great job team!  And thanks @KTanner for posting the final solution back into the thread.

 

High fives all around!

SAS Hackathon registration is open! Build your skills. Make connections. Enjoy creative freedom. Maybe change the world.

Ready to join fellow brilliant minds for the SAS Hackathon?

Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. Registration is now open through August 30th. Visit the SAS Hackathon homepage.

Register today!
What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 8 replies
  • 10357 views
  • 12 likes
  • 3 in conversation