Hello There!
I'm having trouble connecting to a CEPH object storage service using PROC S3 in SAS Viya 3.5 on Linux. CEPH is an open-source distributed storage system capable of providing S3 compatible buckets. I've tried setting a custom region using the TKS3_CUSTOM_REGION environment variable and a config file with access credentials, but I keep getting an error saying "The AWS access key Id you provided does not exist in our records." Here's my code:
options set=TKS3_CUSTOM_REGION="us-east-1,storage.yada.yada,0,0,TRUE,TRUE"; %let myvar_value = %sysget(TKS3_CUSTOM_REGION); %put &myvar_value; PROC S3 config="/home/user/tks3.cfg"; LIST "/"; RUN;
The S3 "/home/user/tks3.cfg"
file contains the following:
region=us-east-1 keyID = thisIsAnAwesomeKeyID secret = whoaThisSecretIsSafe
Output Log:
1 %studio_hide_wrapper; 83 options set=TKS3_CUSTOM_REGION="us-east-1,storage.yada.yada,0,0,TRUE,TRUE"; 84 %let myvar_value = %sysget(TKS3_CUSTOM_REGION); 85 %put &myvar_value; us-east-1,storage.yada.yada,0,0,TRUE,TRUE 86 87 PROC S3 88 config="/home/user/tks3.cfg"; 91 LIST "/"; 92 RUN; ERROR: Could not get user buckets. ERROR: The AWS access key Id you provided does not exist in our records. ERROR: The AWS access key Id you provided does not exist in our records. NOTE: The SAS System stopped processing this step because of errors. NOTE: PROCEDURE S3 used (Total process time): real time 0.58 seconds cpu time 0.02 seconds 93 94 %studio_hide_wrapper; 105 106
Currently I am able to interact with data from the CEPH bucket using the command-line tool for managing Amazon S3 storage, s3cmd, the same way I use it to interact with AWS S3 buckets. I guess this rules out problems caused by the CEPH service.
Can someone point what might be wrong?
Thank you.
Environment info:
SAS Viya 3.5 running on Linux
I managed to resolve the issue by incorporating a custom region using a different approach. Since there was no response, I'll share my solution for anyone who might benefit from it.
Here's what I did to make it work:
/*Add custom region*/
proc s3;
REGION ADD HOST="storage.middle.earth" NAME="shire";
run;
/* Listing all regions (including the custom region) */
proc s3;
region list;
run;
This resulted in the following log output (note the added custom region):1 %studio_hide_wrapper;
83 proc s3;
84 REGION ADD HOST="storage.middle.earth" NAME="shire";
85 run;
NOTE: PROCEDURE S3 used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds
86
87 proc s3;
88 region list;
89 run;
Amazon Regions
us-east-1 s3.amazonaws.com Default 0
us-east-2 s3-us-east-2.amazonaws.com Default 0
us-west-2 s3-us-west-2.amazonaws.com Default 0
us-west-1 s3-us-west-1.amazonaws.com Default 0
eu-west-1 s3-eu-west-1.amazonaws.com Default 0
eu-central-1 s3-eu-central-1.amazonaws.com Default 0
ap-southeast-1 s3-ap-southeast-1.amazonaws.com Default 0
ap-southeast-2 s3-ap-southeast-2.amazonaws.com Default 0
ap-northeast-1 s3-ap-northeast-1.amazonaws.com Default 0
sa-east-1 s3-sa-east-1.amazonaws.com Default 0
us-gov-west-1 s3-us-gov-west-1.amazonaws.com Default 0
us-gov-west-1 s3-fips-us-gov-west-1.amazonaws.com Default 0
ca-central-1 s3-ca-central-1.amazonaws.com Default 0
ap-south-1 s3-ap-south-1.amazonaws.com Default 0
ap-northeast-2 s3-ap-northeast-2.amazonaws.com Default 0
cn-north-1 s3-cn-north-1.amazonaws.com Default 0
cn-northwest-1 s3-cn-northwest-1.amazonaws.com Default 0
eu-west-2 s3-eu-west-2.amazonaws.com Default 0
eu-west-3 s3-eu-west-3.amazonaws.com Default 0
ap-east-1 s3.ap-east-1.amazonaws.com Default 0
eu-south-1 s3.eu-south-1.amazonaws.com Default 0
eu-north-1 s3.eu-north-1.amazonaws.com Default 0
me-south-1 s3.me-south-1.amazonaws.com Default 0
af-south-1 s3.af-south-1.amazonaws.com Default 0
ap-northeast-3 s3.ap-northeast-3.amazonaws.com Default 0
Custom Regions
shire storage.middle.earth Default 0
NOTE: PROCEDURE S3 used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds
90
91 %studio_hide_wrapper;
102
103
Now, with the appropriate credentials, it's possible to test bi listing a bucket content:
proc s3 KEYID="isItSecret" SECRET="isItSafe" region="shire";
list "/bucket" ;
run;
I managed to resolve the issue by incorporating a custom region using a different approach. Since there was no response, I'll share my solution for anyone who might benefit from it.
Here's what I did to make it work:
/*Add custom region*/
proc s3;
REGION ADD HOST="storage.middle.earth" NAME="shire";
run;
/* Listing all regions (including the custom region) */
proc s3;
region list;
run;
This resulted in the following log output (note the added custom region):1 %studio_hide_wrapper;
83 proc s3;
84 REGION ADD HOST="storage.middle.earth" NAME="shire";
85 run;
NOTE: PROCEDURE S3 used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds
86
87 proc s3;
88 region list;
89 run;
Amazon Regions
us-east-1 s3.amazonaws.com Default 0
us-east-2 s3-us-east-2.amazonaws.com Default 0
us-west-2 s3-us-west-2.amazonaws.com Default 0
us-west-1 s3-us-west-1.amazonaws.com Default 0
eu-west-1 s3-eu-west-1.amazonaws.com Default 0
eu-central-1 s3-eu-central-1.amazonaws.com Default 0
ap-southeast-1 s3-ap-southeast-1.amazonaws.com Default 0
ap-southeast-2 s3-ap-southeast-2.amazonaws.com Default 0
ap-northeast-1 s3-ap-northeast-1.amazonaws.com Default 0
sa-east-1 s3-sa-east-1.amazonaws.com Default 0
us-gov-west-1 s3-us-gov-west-1.amazonaws.com Default 0
us-gov-west-1 s3-fips-us-gov-west-1.amazonaws.com Default 0
ca-central-1 s3-ca-central-1.amazonaws.com Default 0
ap-south-1 s3-ap-south-1.amazonaws.com Default 0
ap-northeast-2 s3-ap-northeast-2.amazonaws.com Default 0
cn-north-1 s3-cn-north-1.amazonaws.com Default 0
cn-northwest-1 s3-cn-northwest-1.amazonaws.com Default 0
eu-west-2 s3-eu-west-2.amazonaws.com Default 0
eu-west-3 s3-eu-west-3.amazonaws.com Default 0
ap-east-1 s3.ap-east-1.amazonaws.com Default 0
eu-south-1 s3.eu-south-1.amazonaws.com Default 0
eu-north-1 s3.eu-north-1.amazonaws.com Default 0
me-south-1 s3.me-south-1.amazonaws.com Default 0
af-south-1 s3.af-south-1.amazonaws.com Default 0
ap-northeast-3 s3.ap-northeast-3.amazonaws.com Default 0
Custom Regions
shire storage.middle.earth Default 0
NOTE: PROCEDURE S3 used (Total process time):
real time 0.00 seconds
cpu time 0.00 seconds
90
91 %studio_hide_wrapper;
102
103
Now, with the appropriate credentials, it's possible to test bi listing a bucket content:
proc s3 KEYID="isItSecret" SECRET="isItSafe" region="shire";
list "/bucket" ;
run;
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.
Find more tutorials on the SAS Users YouTube channel.