topic Re: JSON input in proc http in SAS Programming

JSON input in proc http

SGSP — Mon, 10 May 2021 14:05:20 GMT

Hello,

I've searched around for a similar problem, so if I've missed something, please let me know.

I'm trying to make proc http work with an restAPI. I am able to do it via curl and Python, so the API itself should be fine. Since this is a local web-service, it'll be hard to replicate the error, but let me know if there's anything I can do to make the description more thorough.

I'm using SAS EG 8.1. This is what I've tried so far:

Attempt number 1:

filename json_in "C:\Temp\json_in.json";
data _null_;
file json_in;
put '{';
put '"valueDate": "2021-05-10T04:32:34.394Z",';
put '"inputSpecification": "TESTINP",';
put '"name": "FOO",';
put '"parameters": [';
put '{';
put '"name": "Parameter1",';
put '"title": "My Parameter",';
put '"value": "coverstat1"';
put '}';
put '],';
put '"configuration": "TESTCONF",';
put '"scenarios": [';
put '"Scen1"';
put '],';
put '"outputDestination": "Database"';
put '}';
run;
filename out "C:\temp\out.txt";
proc http url="http://blah/api/v2/Batches" 
METHOD="POST"
in="C:\temp\json_in.json"
ct="application/json"
out=out;
headers "accept"="application/json" 
/*"Content-Type"="application/json"*/;
run;

Attempt number 2:

proc http METHOD="POST" 
url="http://blah/api/v2/Batches" 
ct="application/json"
in="{ ""valueDate"": ""2021-05-10T04:32:34.394Z"", ""inputSpecification"": ""TESTINP"", ""name"": ""FOO"", ""configuration"": ""TESTCONF"", ""scenarios"": {[""SCEN1""]}, ""parameters"": [{""name"": ""Parameter1"", ""title"": ""MyParameter"", ""value"": ""coverstat1""}]"
ct="application/json"
out=out;
headers "accept"="application/json";
run;

Both of these attempt yield the same error from the API:

{"detail":"Value cannot be null.\r\nParameter name: source"}

Apparently this has to do with the parameter-input being multi-dimensional. Is there any way of handling this in SAS?

For completeness, this is the Python-code that works:

import requests

params = {
  "valueDate": "2021-05-10T04:32:34.394Z",
  "inputSpecification": "TESTINP",
  "name": "FOO",
  "parameters": [
    {
      "name": "Parameter1",
      "title": "My Parameter",
      "value": "coverstat1"
    }
  ],
  "configuration": "TESTCONF",
  "scenarios": [
    "SCEN1"
  ],
  "outputDestination": "Database"
}
url = 'http://blah/api/v2/Batches'
headers = {'accept': 'application/json'}
response = requests.post(url, json=params, headers=headers)

print(response.json())

I know it's not ideal, that the webservice can't be accessed by people here, but hopefully I've done enough to describe the problem.

Thanks in advance.

Re: JSON input in proc http

ChrisHemedinger — Mon, 10 May 2021 14:45:32 GMT

The IN= accepts either a fileref (no quotes, not a literal filename) or the content of the payload you want to POST. So your first example should be:

proc http url="http://blah/api/v2/Batches" METHOD="POST" in=json_in ct="application/json" out=out; headers "accept"="application/json" run;

Re: JSON input in proc http

Oligolas — Tue, 11 May 2021 09:02:18 GMT

Hi,

the code itself seems to work; when I'm using url="http://httpbin.org/put" I get no errors.

correction: url="http://httpbin.org/post"

Re: JSON input in proc http

SGSP — Mon, 10 May 2021 16:05:44 GMT

Thanks for your reply. I did try with the direct reference to json_in as well. Still the same error.

Re: JSON input in proc http

SGSP — Tue, 11 May 2021 06:08:11 GMT

This is weird. When I do the same thing, I get the following reply:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>405 Method Not Allowed</title>
<h1>Method Not Allowed</h1>
<p>The method is not allowed for the requested URL.</p>

I'm guessing this also has to do with the multi-dimensional parameters input. Could it be something to do with the version I'm running? (SAS EG 8.1)

Re: JSON input in proc http

Kurt_Bremser — Tue, 11 May 2021 08:42:13 GMT

That is a message you get from the website. The particular URL does not allow a PUT method for the HTTP request. It may be that you use the URL of a page where the actual PUT request is triggered through a button on that page.

Re: JSON input in proc http

Oligolas — Tue, 11 May 2021 09:01:27 GMT

Sorry my bad the URL for test should be

url="http://httpbin.org/post"

Re: JSON input in proc http

SGSP — Tue, 11 May 2021 12:48:53 GMT

Thanks for showing me that website, I'm not used to working with SAS or APIs, and that'll be a big help.

I still haven't found a solution, however, but I have a bit more information. When I run the following

filename out "\\sas00750\analyse3\AKF\Data\Personlige\SIG\out.txt";
filename json_in "\\sas00750\analyse3\AKF\Data\Personlige\SIG\json_in.json";
data _null_;
file json_in recfm=f lrecl=1;
put '{"valueDate": "2021-05-10T04:32:34.394Z", "inputSpecification": "TESTINP", "name": "FOO", "parameters": [{"name": "Parameter1", "title": "My Parameter", "value": "coverstat1"}], "configuration": "TESTCONF", "scenarios": ["SCEN1"], "outputDestination": "Database"}';
run;
proc http METHOD="POST" 
url="http://httpbin.org/post" 
ct="application/json"
in=json_in 
ct="application/json"
out=out;
headers "accept"="application/json";
run;

I get this response:

{
  "args": {}, 
  "data": "{\"valueDate\": \"2021-05-10T04:32:34.394Z\", \"inputSpecification\": \"TESTINP\", \"name\": \"FOO\", \"parameters\": [{\"name\": \"Parameter1\", \"title\": \"My Parameter\", \"value\": \"coverstat1\"}], \"configuration\": \"TESTCONF\", \"scenarios\": [\"SCEN1\"], \"outputDestination\": \"Database\"}", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "application/json", 
    "Content-Length": "263", 
    "Content-Type": "application/json", 
    "Host": "httpbin.org", 
    "User-Agent": "SAS/9", 
    "X-Amzn-Trace-Id": "Root=1-609a7799-4b92fc8f27fb3ea812533140"
  }, 
  "json": {
    "configuration": "TESTCONF", 
    "inputSpecification": "TESTINP", 
    "name": "FOO", 
    "outputDestination": "Database", 
    "parameters": [
      {
        "name": "Parameter1", 
        "title": "My Parameter", 
        "value": "coverstat1"
      }
    ], 
    "scenarios": [
      "SCEN1"
    ], 
    "valueDate": "2021-05-10T04:32:34.394Z"
  }, 
  "origin": "", 
  "url": "http://httpbin.org/post"
}

However, when I run the Python code that works (see original post) I get this:

{
  'args': {}, 
  'data': '{"valueDate": "2021-05-10T04:32:34.394Z", "inputSpecification": "TESTINP", "name": "FOO", "parameters": [{"name": "Parameter1", "title": "My Parameter", "value": "coverstat1"}], "configuration": "TESTCONF", "scenarios": ["SCEN1"], "outputDestination": "Database"}', 
  'files': {}, 
  'form': {}, 
  'headers': {
    'Accept': 'application/json', 
    'Accept-Encoding': 'gzip, deflate', 
    'Content-Length': '263', 
    'Content-Type': 'application/json', 
    'Host': 'httpbin.org', 
    'User-Agent': 'python-requests/2.25.1', 
    'X-Amzn-Trace-Id': 'Root=1-609a7993-516acbb91c5c1f902ee96668'
  }, 
  'json': {
    'configuration': 'TESTCONF', 
    'inputSpecification': 'TESTINP', 
    'name': 'FOO', 
    'outputDestination': 'Database', 
    'parameters': [
      {
        'name': 'Parameter1', 
        'title': 'My Parameter', 
        'value': 'coverstat1'
      }
    ], 
    'scenarios': [
      'SCEN1'
    ], 
    'valueDate': '2021-05-10T04:32:34.394Z'
  }, 
  'origin': '', 
  'url': 'http://httpbin.org/post'
}

Note: When I replace the single quotes with double quotes and vice versa, in the SAS program, then it doesn't seem like httpbin interprets it as json.

I'm wondering if it could be an encoding issue, since the python request includes 'Accept-Encoding': 'gzip, deflate'? Is there a similar option to set in SAS?

Thanks for your help.

Re: JSON input in proc http

ChrisHemedinger — Tue, 11 May 2021 13:05:35 GMT

You already have a HEADERS statement, and you could add to that:

headers "accept"="application/json" "Accept-Encoding"="gzip, deflate";

Re: JSON input in proc http

SGSP — Tue, 11 May 2021 13:59:06 GMT

Thanks 🙂

This unfortunately doesn't solve the problem. But now the only remaining difference seems to be the double quotes/single quotes. Perhaps the application only accepts the single quotes. Does anyone have an idea as to why the response from httpbin is in single quotes when I use Python or curl and in double quotes when I use SAS?

Re: JSON input in proc http

Oligolas — Tue, 11 May 2021 14:12:35 GMT

JSON is standardized to use double-quotes, to the degree that this would not be considered standard JSON and would error in a JSON parser.

Re: JSON input in proc http

SGSP — Tue, 11 May 2021 14:47:21 GMT

That's true, but what confuses me is that the response from httpbin is in single-quotes when I use Python or curl, even though my json input is in double-quotes. Then, when I use SAS, the response is in double-quotes.

Re: JSON input in proc http

Oligolas — Tue, 11 May 2021 15:50:34 GMT

It's confusing to me too

Re: JSON input in proc http

Tom — Tue, 11 May 2021 16:50:03 GMT

I don't see any parameter named "source" in your JSON.

Are you sure you are sending the same JSON as you send with the other methods?

Re: JSON input in proc http

SGSP — Tue, 11 May 2021 17:23:43 GMT

If I have to be completely honest, I’m not entirely sure about anything. I’m a bit of a novice with both SAS and APIs.

But going by the responses from httpbin that I poster earlier, it seems like the only difference between the response for the Python request and the response for the SAS request is the quotation marks. At least after I added the header.

Re: JSON input in proc http

SGSP — Tue, 11 May 2021 17:26:38 GMT

Oh, and the data line as well. I have no idea if those two differences are connected to each other.