Hello,
I have found this community to be very helpful and have another issue I am working through and looking for some guidance. I am trying to pull in some data from Shopify that uses cursor based pagination. I have 1K+ products I need to pull in and looking for a way to essentially loop through the pages and not just pull in the initial amount which is 50.
What I have gathered so far is I need to somehow reference the link headers that look something like this: "Link: <https://tXXXXXXXXXXXX.myshopify.com/admin/api/2021-10/products.json?limit=50&page_info=eyJsYXN0X2lkI...
YXN0X3ZhbHVlIjoiQmFqYSBEZXNpZ25zIExQOSBTcG9ydCBMRUQgTGlnaHQiLCJkaXJlY3Rpb24iOiJuZXh0In0>; rel="next"" (see: https://shopify.dev/api/usage/pagination-rest ) I was able to generate the link headers in the log using the "headerout=headers" statement shown in my code below. How can I then take that sort of information (specifically the "rel="next") and use it to then pull the next page. Or perhaps there is a much easier way to do this? For other APIs I have been able to loop through the pages, but that doesn't appear to be an option here.
Here is my code that works, but only pulls the first 50:
%let accesskey=XXXXXXXXXX;
%let fullurl=https://XXXXXXX.myshopify.com/admin/api/2021-10/products.json;
filename tb_prods temp;
filename headers temp;
proc http
url= "&fullurl"
method="GET" out=tb_prods headerout=headers;
headers
"X-Shopify-Access-Token"="&accesskey.";
run;
data _null_;
infile headers;
input;
put _infile_;
run;
Any help would be greatly appreciated.
Can you try?
%let fullurl=https://XXXXXXX.myshopify.com/admin/api/2021-10/products.json?limit=2000;
Unfortunately, Shopify limits the product API calls to 250 or less.
So the best I can do with one API call is:
%let fullurl=https://XXXXXXXX.myshopify.com/admin/api/2021-10/products.json?limit=250;
I am needing to do cursor based pagination. Right now this is printing in the log:
Link: <https://XXXXXX.myshopify.com/admin/api/2021-10/products.json limit=250&page_info=eyJsYXN0X2lkIjo2NjU0NjU1NjYwMDc4LCJsYXN0X3ZhbHVlIjoiQ2FsaSBSYWlzZWQgTG93ZXIgQ29udHJvbCBBcm0gU2tpZCBQbGF0ZSAtIFRveW90YSBUYWNvbWEgKDIwMDUtMjAyMSkiLCJkaXJlY3Rpb24iOiJuZXh0In0>; rel="next"
I am basically needing to reference the "rel="next" in my code to tell SAS to pull the next 250 products.
I wanted to follow-up on my previous message. Can anyone explain the best way to make a paginated request in SAS. Here is the documentation I am just not sure how to translate this into my SAS code.
https://shopify.dev/api/usage/pagination-rest
It looks like I need to somehow access the rel="next" that prints in the headers file to tell SAS to pull the next set of products.
Catch the best of SAS Innovate 2025 — anytime, anywhere. Stream powerful keynotes, real-world demos, and game-changing insights from the world’s leading data and AI minds.
Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.
Find more tutorials on the SAS Users YouTube channel.
Ready to level-up your skills? Choose your own adventure.