Hello,
I have found this community to be very helpful and have another issue I am working through and looking for some guidance. I am trying to pull in some data from Shopify that uses cursor based pagination. I have 1K+ products I need to pull in and looking for a way to essentially loop through the pages and not just pull in the initial amount which is 50.
What I have gathered so far is I need to somehow reference the link headers that look something like this: "Link: <https://tXXXXXXXXXXXX.myshopify.com/admin/api/2021-10/products.json?limit=50&page_info=eyJsYXN0X2lkI...
YXN0X3ZhbHVlIjoiQmFqYSBEZXNpZ25zIExQOSBTcG9ydCBMRUQgTGlnaHQiLCJkaXJlY3Rpb24iOiJuZXh0In0>; rel="next"" (see: https://shopify.dev/api/usage/pagination-rest ) I was able to generate the link headers in the log using the "headerout=headers" statement shown in my code below. How can I then take that sort of information (specifically the "rel="next") and use it to then pull the next page. Or perhaps there is a much easier way to do this? For other APIs I have been able to loop through the pages, but that doesn't appear to be an option here.
Here is my code that works, but only pulls the first 50:
%let accesskey=XXXXXXXXXX;
%let fullurl=https://XXXXXXX.myshopify.com/admin/api/2021-10/products.json;
filename tb_prods temp;
filename headers temp;
proc http
url= "&fullurl"
method="GET" out=tb_prods headerout=headers;
headers
"X-Shopify-Access-Token"="&accesskey.";
run;
data _null_;
infile headers;
input;
put _infile_;
run;
Any help would be greatly appreciated.
Can you try?
%let fullurl=https://XXXXXXX.myshopify.com/admin/api/2021-10/products.json?limit=2000;
Unfortunately, Shopify limits the product API calls to 250 or less.
So the best I can do with one API call is:
%let fullurl=https://XXXXXXXX.myshopify.com/admin/api/2021-10/products.json?limit=250;
I am needing to do cursor based pagination. Right now this is printing in the log:
Link: <https://XXXXXX.myshopify.com/admin/api/2021-10/products.json limit=250&page_info=eyJsYXN0X2lkIjo2NjU0NjU1NjYwMDc4LCJsYXN0X3ZhbHVlIjoiQ2FsaSBSYWlzZWQgTG93ZXIgQ29udHJvbCBBcm0gU2tpZCBQbGF0ZSAtIFRveW90YSBUYWNvbWEgKDIwMDUtMjAyMSkiLCJkaXJlY3Rpb24iOiJuZXh0In0>; rel="next"
I am basically needing to reference the "rel="next" in my code to tell SAS to pull the next 250 products.
I wanted to follow-up on my previous message. Can anyone explain the best way to make a paginated request in SAS. Here is the documentation I am just not sure how to translate this into my SAS code.
https://shopify.dev/api/usage/pagination-rest
It looks like I need to somehow access the rel="next" that prints in the headers file to tell SAS to pull the next set of products.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.
Find more tutorials on the SAS Users YouTube channel.