Hi experts,
I'm pulling some data from a REST API using proc HTTP and saving the response into a JSON file, I'm trying to parse this file using many libname, datasteps etc to build a generalized parser.
I already have a python code that uses pandas and parses the same json file.
I'm attaching the response file and the python file, does anyone have any idea how to "translate" this python code into SAS? or use libname correctly.
Thanks for the help
Yes, I'm in SAS 9.4
For Base SAS Software ... Custom version information: 9.4_M6 Image version information: 9.04.01M6P110718 For SAS/STAT ... Custom version information: 15.1 For SAS/ETS ... Custom version information: 15.1 For SAS/IML ... Custom version information: 15.1 For High Performance Suite ... Custom version information: 2.2_M7 For SAS/ACCESS Interface to PC Files ... Custom version information: 9.4_M6 NOTE: PROCEDURE PRODUCT_STATUS used (Total process time): real time 0.01 seconds cpu time 0.02 seconds
SAS can already read JSON files.
%let fname=ReportResponse.txt;
filename map temp;
filename resp "&path/&fname";
libname resp json jsonmap=map automap=reuse ;
proc copy inlib=resp outlib=work;
run;
NOTE: Copying RESP.ALLDATA to WORK.ALLDATA (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 817 observations read from the data set RESP.ALLDATA. NOTE: The data set WORK.ALLDATA has 817 observations and 11 variables. NOTE: Copying RESP.ATTRIBUTES_FORMS to WORK.ATTRIBUTES_FORMS (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 3 observations read from the data set RESP.ATTRIBUTES_FORMS. NOTE: The data set WORK.ATTRIBUTES_FORMS has 3 observations and 6 variables. NOTE: Copying RESP.CHILDREN_CHILDREN to WORK.CHILDREN_CHILDREN (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 34 observations read from the data set RESP.CHILDREN_CHILDREN. NOTE: The data set WORK.CHILDREN_CHILDREN has 34 observations and 3 variables. NOTE: Copying RESP.CHILDREN_ELEMENT to WORK.CHILDREN_ELEMENT (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 8 observations read from the data set RESP.CHILDREN_ELEMENT. NOTE: The data set WORK.CHILDREN_ELEMENT has 8 observations and 5 variables. NOTE: Copying RESP.CHILDREN_ELEMENT2 to WORK.CHILDREN_ELEMENT2 (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 34 observations read from the data set RESP.CHILDREN_ELEMENT2. NOTE: The data set WORK.CHILDREN_ELEMENT2 has 34 observations and 5 variables. NOTE: Copying RESP.DATA_PAGING to WORK.DATA_PAGING (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 1 observations read from the data set RESP.DATA_PAGING. NOTE: The data set WORK.DATA_PAGING has 1 observations and 8 variables. NOTE: Copying RESP.DATA_ROOT to WORK.DATA_ROOT (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 1 observations read from the data set RESP.DATA_ROOT. NOTE: The data set WORK.DATA_ROOT has 1 observations and 3 variables. NOTE: Copying RESP.DEFINITION_ATTRIBUTES to WORK.DEFINITION_ATTRIBUTES (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 2 observations read from the data set RESP.DEFINITION_ATTRIBUTES. NOTE: The data set WORK.DEFINITION_ATTRIBUTES has 2 observations and 5 variables. NOTE: Copying RESP.DEFINITION_METRICS to WORK.DEFINITION_METRICS (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 2 observations read from the data set RESP.DEFINITION_METRICS. NOTE: The data set WORK.DEFINITION_METRICS has 2 observations and 8 variables. NOTE: Copying RESP.DEFINITION_SORTING to WORK.DEFINITION_SORTING (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 2 observations read from the data set RESP.DEFINITION_SORTING. NOTE: The data set WORK.DEFINITION_SORTING has 2 observations and 4 variables. NOTE: Copying RESP.DEFINITION_THRESHOLDS to WORK.DEFINITION_THRESHOLDS (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 2 observations read from the data set RESP.DEFINITION_THRESHOLDS. NOTE: The data set WORK.DEFINITION_THRESHOLDS has 2 observations and 6 variables. NOTE: Copying RESP.ELEMENT_FORMVALUES to WORK.ELEMENT_FORMVALUES (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 8 observations read from the data set RESP.ELEMENT_FORMVALUES. NOTE: The data set WORK.ELEMENT_FORMVALUES has 8 observations and 3 variables. NOTE: Copying RESP.ELEMENT_FORMVALUES2 to WORK.ELEMENT_FORMVALUES2 (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 34 observations read from the data set RESP.ELEMENT_FORMVALUES2. NOTE: The data set WORK.ELEMENT_FORMVALUES2 has 34 observations and 4 variables. NOTE: Copying RESP.METRICS_NUMBERFORMATTING to WORK.METRICS_NUMBERFORMATTING (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 2 observations read from the data set RESP.METRICS_NUMBERFORMATTING. NOTE: The data set WORK.METRICS_NUMBERFORMATTING has 2 observations and 9 variables. NOTE: Copying RESP.METRICS_RANK_BY_REGION to WORK.METRICS_RANK_BY_REGION (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 34 observations read from the data set RESP.METRICS_RANK_BY_REGION. NOTE: The data set WORK.METRICS_RANK_BY_REGION has 34 observations and 5 variables. NOTE: Copying RESP.METRICS_SALES to WORK.METRICS_SALES (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 34 observations read from the data set RESP.METRICS_SALES. NOTE: The data set WORK.METRICS_SALES has 34 observations and 6 variables. NOTE: Copying RESP.METRICS_THRESHOLDS to WORK.METRICS_THRESHOLDS (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 1 observations read from the data set RESP.METRICS_THRESHOLDS. NOTE: The data set WORK.METRICS_THRESHOLDS has 1 observations and 4 variables. NOTE: Copying RESP.ROOT to WORK.ROOT (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 1 observations read from the data set RESP.ROOT. NOTE: The data set WORK.ROOT has 1 observations and 5 variables. NOTE: Copying RESP.ROOT_CHILDREN to WORK.ROOT_CHILDREN (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 8 observations read from the data set RESP.ROOT_CHILDREN. NOTE: The data set WORK.ROOT_CHILDREN has 8 observations and 4 variables. NOTE: Copying RESP.SORTING_ATTRIBUTE to WORK.SORTING_ATTRIBUTE (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 1 observations read from the data set RESP.SORTING_ATTRIBUTE. NOTE: The data set WORK.SORTING_ATTRIBUTE has 1 observations and 4 variables. NOTE: Copying RESP.SORTING_FORM to WORK.SORTING_FORM (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 1 observations read from the data set RESP.SORTING_FORM. NOTE: The data set WORK.SORTING_FORM has 1 observations and 4 variables. NOTE: Copying RESP.SORTING_METRIC to WORK.SORTING_METRIC (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 1 observations read from the data set RESP.SORTING_METRIC. NOTE: The data set WORK.SORTING_METRIC has 1 observations and 4 variables. NOTE: Copying RESP.SOURCE to WORK.SOURCE (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 1 observations read from the data set RESP.SOURCE. NOTE: The data set WORK.SOURCE has 1 observations and 3 variables. NOTE: Copying RESP.THRESHOLDS_FORMAT to WORK.THRESHOLDS_FORMAT (memtype=DATA). NOTE: BUFSIZE is not cloned when copying across different engines. System Option for BUFSIZE was used. NOTE: There were 2 observations read from the data set RESP.THRESHOLDS_FORMAT. NOTE: The data set WORK.THRESHOLDS_FORMAT has 2 observations and 4 variables. NOTE: PROCEDURE COPY used (Total process time): real time 0.37 seconds cpu time 0.17 seconds
It works partially, it does not parse in the desired format.
I need to extract attributes and it's values and then metrics to transform in a dataset where I can work with.
I've attached the desired result from the json
It might be possible to craft your own map file to pass to the JSON engine.
Or write code to re-merge the tables that the default method creates.
But then it is not longer "generic".
Really hard to follow what the question is when you hide the details in attachments.
It looks like you just want to extract these "fields" from the JSON data.
Region|Employee|Employee2|Sales|Rank_by_Region Northeast|Sawyer|Leanne|$2,411,912|1 Northeast|Kelly|Laura|$2,350,720|2
Not sure if you do it with JSON map but just pulling together some of the fields found by the generic mapping I can get something like your table. Not sure if it is pulling the fields you want and whether it is pulling ALL of the values you want.
filename map2 temp;
data _null_;
file map2;
put '
{
"DATASETS": [
{
"DSNAME": "WANT",
"TABLEPATH": "/root/result/data/root/children/children/metrics/Rank by Region",
"VARIABLES": [
{
"NAME": "Region",
"TYPE": "CHARACTER",
"PATH": "/root/result/data/root/children/element/name",
"CURRENT_LENGTH": 12,
"OPTIONS" : ["RETAIN"]
},
{
"NAME": "Last_Name",
"TYPE": "CHARACTER",
"PATH": "/root/result/data/root/children/children/element/formValues/Last Name",
"CURRENT_LENGTH": 11
},
{
"NAME": "First_Name",
"TYPE": "CHARACTER",
"PATH": "/root/result/data/root/children/children/element/formValues/First Name",
"CURRENT_LENGTH": 8
},
{
"NAME": "Sales",
"TYPE": "CHARACTER",
"PATH": "/root/result/data/root/children/children/metrics/Sales/fv",
"CURRENT_LENGTH": 10
},
{
"NAME": "Rank",
"TYPE": "NUMERIC",
"PATH": "/root/result/data/root/children/children/metrics/Rank by Region/rv"
}
]
}
]
}
';
run;
%let path=c:\downloads;
%let fname=ReportResponse.txt;
filename resp "&path/&fname";
libname resp json map=map2 ;
options nocenter ls=132 ps=100; run;
proc print data=resp.want;
run;
First_ Obs Region Last_Name Name Sales Rank 1 Northeast Sawyer Leanne $2,411,912 1 2 Northeast Kelly Laura $2,350,720 2 3 Northeast Yager Beth $2,303,847 3 4 Northeast De Le Torre Sandra $607,895 4 5 Northeast Kieferson Jack $584,933 5 6 Northeast Sonder Melanie $295,108 6 7 Mid-Atlantic Bernstein Lawrence $1,060,632 1 8 Mid-Atlantic Folks Adrienne $1,047,776 2 9 Mid-Atlantic Hollywood Robert $1,026,874 3 10 Mid-Atlantic Brown Vernon $331,735 4 11 Mid-Atlantic Corcoran Peter $325,147 5 12 Mid-Atlantic Ingles Walter $229,439 6 13 Mid-Atlantic Smith Thomas $221,379 7 14 Mid-Atlantic Young Sarah $209,634 8 15 Southeast Strome Fred $595,372 1 16 Southeast Lynch Sam $592,471 2 17 Southeast McClain Sean $531,371 3 18 Southeast Benner Ian $520,737 4 19 Central Torrison Mary $1,690,350 1 20 Central Gale Loren $1,669,290 2 21 Central Ellerkamp Nancy $847,227 3 22 Central Zemlicka George $822,500 4 23 South Pierce Charles $2,084,241 1 24 South Nelson Arthur $1,654,297 2 25 South Conner Beatrice $1,650,742 3 26 Northwest Gedot Harriet $739,741 1 27 Northwest Hall David $513,213 2 28 Northwest Becker Kyle $508,234 3 29 Southwest Bates Michael $1,068,907 1 30 Southwest Bell Caitlin $1,040,481 2 31 Southwest Hunt Matthew $731,413 3 32 Southwest Johnson Andrew $445,052 4 33 Southwest Schafer Rose $408,280 5 34 Web Walker Robert $3,902,762 1
Note that you can also tell it how to process the fields. For example you might want SALES to be numeric instead of character.
{
"NAME": "Sales",
"TYPE": "NUMERIC",
"PATH": "/root/result/data/root/children/children/metrics/Sales/fv",
"INFORMAT" : [ "COMMA", 32, 0 ],
"FORMAT": ["DOLLAR",10,0]
}
Registration is now open for SAS Innovate 2025 , our biggest and most exciting global event of the year! Join us in Orlando, FL, May 6-9.
Sign up by Dec. 31 to get the 2024 rate of just $495.
Register now!
SAS' Charu Shankar shares her PROC SQL expertise by showing you how to master the WHERE clause using real winter weather data.
Find more tutorials on the SAS Users YouTube channel.