Hello,
I have found this community to be very helpful and have another issue I am working through and looking for some guidance. I am trying to pull in some data from Shopify that uses cursor based pagination. I have 1K+ products I need to pull in and looking for a way to essentially loop through the pages and not just pull in the initial amount which is 50.
What I have gathered so far is I need to somehow reference the link headers that look something like this: "Link: <https://tXXXXXXXXXXXX.myshopify.com/admin/api/2021-10/products.json?limit=50&page_info=eyJsYXN0X2lkI...
YXN0X3ZhbHVlIjoiQmFqYSBEZXNpZ25zIExQOSBTcG9ydCBMRUQgTGlnaHQiLCJkaXJlY3Rpb24iOiJuZXh0In0>; rel="next"" (see: https://shopify.dev/api/usage/pagination-rest ) I was able to generate the link headers in the log using the "headerout=headers" statement shown in my code below. How can I then take that sort of information (specifically the "rel="next") and use it to then pull the next page. Or perhaps there is a much easier way to do this? For other APIs I have been able to loop through the pages, but that doesn't appear to be an option here.
Here is my code that works, but only pulls the first 50:
%let accesskey=XXXXXXXXXX;
%let fullurl=https://XXXXXXX.myshopify.com/admin/api/2021-10/products.json;
filename tb_prods temp;
filename headers temp;
proc http
url= "&fullurl"
method="GET" out=tb_prods headerout=headers;
headers
"X-Shopify-Access-Token"="&accesskey.";
run;
data _null_;
infile headers;
input;
put _infile_;
run;
Any help would be greatly appreciated.
