DATA Step, Macro, Functions and more

Downloading a file from AWS S3 (Amazon Web Services) using SAS and cURL

Reply
Contributor jp
Contributor
Posts: 29

Downloading a file from AWS S3 (Amazon Web Services) using SAS and cURL

Hello all, I'm trying to get SAS to call cURL, to download a file from AWS.

 

I'm currently getting a "AWS authentication requires a valid Date or x-amz-date header" error, but I have checked the date (Fri, 29 JAN 2016 14:47:15 GMT+1300) and it seems to be in the right format so I'm not sure what is happening. When I was testing yesterday with what I thought was the same code I got an error along the lines of "Problem with your signed signature".

 

After a good day of attempts I'm hoping someone else may have done something similar and can help. Here's my code I'm using currently. I will mention I'm not an AWS expert so it's possible I've messed something basic up but I haven't been able to figure it out yet.

 

 

* cludged from:
http://stackoverflow.com/questions/27658147/download-private-file-from-s3-using-bash
https://communities.sas.com/t5/Base-SAS-Programming/Using-sas-secure-to-generate-hmac-sha1-oauth-signature/td-p/169922 ;


%LET cUrl_exe = C:\Apps\cUrl\cUrl.exe ; * change if yours is in a different place ;
%LET s3Key=<access_key>;* change this line ;
%LET s3Secret=<secret_key>;* change this line ;
%LET out_file = <outfile_path>; * change this line ;
%LET file = test.txt ; * change this line ;
%LET bucket = <bucket name>; * change this line ; 
%LET resource = "/&bucket./&file." ; 
%PUT Resource = [&Resource.] ; 
%LET contentType = 'text/html' ; * this may not matter? ;
*dateValue="`date +'%a, %d %b %Y %H:%M:%S %z'`" ; PROC FORMAT ; PICTURE HTTPTime (DEFAULT = 29) OTHER = '%a, %0d %b %Y %0H:%0M:%0S' (DATATYPE = DATETIME) ; RUN ; DATA _NULL_ ; CALL SYMPUT("dateValue", STRIP(PUT(DATETIME(), HTTPTime.)) || " GMT+1300") ; * SHOULD BE ABLE TO USE %Z TO GET UTC OFFSET NEED TO CHANGE TO +1200 WHEN NOT IN DST ; RUN ; %PUT dateValue = [&dateValue.] ; DATA _NULL_ ; stringToSign = '"GET' || '0A'X || '0A'X || /* STRIP(SYMGET("contentType")) || '0A'X ||*/ STRIP(SYMGET("dateValue")) || '0A'X || STRIP(SYMGET("resource")) || '"' ; PUT stringToSign= ; CALL SYMPUT("stringToSign", STRIP(TRANWRD(stringToSign, '"', '""'))) ; RUN ; proc groovy; add sasjar="commons_codec" ; *version="1.7.0.0_SAS_20121211183158";*version may differ based on your installation. Check your SAS Versioned Jar Repository; submit "&s3secret." "&stringToSign."; import javax.crypto.Mac import javax.crypto.spec.SecretKeySpec import org.apache.commons.codec.binary.Base64 def base64hmacsha256(key, message) { mac = Mac.getInstance("HmacSHA256") mac.init(new SecretKeySpec(key.getBytes(), "HmacSHA256")) sha1_bytes = mac.doFinal(message.getBytes()) base64 = new Base64() return new String(base64.encode(sha1_bytes)) } exports.base64hmacsha256 = base64hmacsha256(args[0], args[1]) endsubmit; quit; %put &base64hmacsha256.; DATA _NULL_ ; LENGTH cmd $1024 ; cmd = STRIP(SYMGET("cUrl_exe")) || ' -U '||"<user:pass>"||' -x <proxy:port> -k'||' -o "' || STRIP(SYMGET("out_file")) || '" -H "Host: ' || STRIP(SYMGET("bucket")) || '.s3.amazonaws.com' || /* '" -H "Content-Type: ' || STRIP(SYMGET("contentType")) || */ '" -H "Date: '||"'"|| STRIP(SYMGET("dateValue")) ||"'"|| '" -H "Authorization: AWS ' || STRIP(SYMGET("s3Key")) || ':' || STRIP(SYMGET("base64hmacsha256")) || '" https://' || STRIP(SYMGET("bucket")) || '.s3.amazonaws.com/' || STRIP(SYMGET("file")) ; RC = SYSTEM(cmd) ; PUT cmd= RC= ; RUN ;

 

The output from this code looks like this:

cmd=C:\Apps\cUrl\cUrl.exe -U <userSmiley Tongueass> -x <proxySmiley Tongueort> -k -o "<outfile_path>" -H "Host: <bucket_name>.s3.amazonaws.com" -H "Date: 'Fri, 29 JAN 2016 14:47:15 GMT+1300'" -H "Authorization: AWS <access_key>:<signed_signature>" https://<bucket_name>.s3.amazonaws.com/test.txt

 

Cheers for any help!

Ask a Question
Discussion stats
  • 0 replies
  • 789 views
  • 0 likes
  • 1 in conversation