topic transfert data from sas to s3 in SAS Data Management

transfert data from sas to s3

demiter — Tue, 27 Aug 2019 08:31:33 GMT

hello
is there a way to transfert data from sas to aws s3 bucket?

Re: transfert data from sas to s3

LinusH — Tue, 27 Aug 2019 14:45:19 GMT

I think the most straigthforward way is to use PROC S3.

https://documentation.sas.com/?docsetId=proc&docsetVersion=9.4&docsetTarget=p1bxmq7xu9hph8n17dmj32sdzoji.htm&locale=en

Re: transfert data from sas to s3

Patrick — Wed, 28 Aug 2019 02:38:00 GMT

@demiter

What's going to work for you will depend on your SAS version. I believe Proc S3 has only been introduced with SAS 9.4 M4

If you're using SAS Viya then you can also access S3 via a caslib statement.

https://www.sas.com/content/dam/SAS/support/en/sas-global-forum-proceedings/2019/3391-2019.pdf

And last but not least: If you've got the new Lustre FSx under AWS available then there might also be an option to interact with S3 via accessing Lustre. This would then just require a normal libname for the Lustre file system (once S3 is "mirrored" onto Lustre).

https://aws.amazon.com/fsx/lustre/

Re: transfert data from sas to s3

Patrick — Wed, 28 Aug 2019 01:56:25 GMT

@demiter

another option is to access S3 via Impala. You then execute your SAS queries against Impala.

Configuring the Amazon S3 Connector

You can securely configure your cluster to authenticate with Amazon Simple Storage Service (S3) using the Cloudera S3 Connector Service. This configuration enables Impala queries to access data in S3 and also enables the Hue S3 Browser. Impala and Hue are automatically configured to authenticate with S3, but applications such as YARN, MapReduce, or Spark must provide their own AWS credentials when submitting jobs. You can define only one Amazon S3 service for each cluster.

https://www.cloudera.com/documentation/enterprise/5-15-x/topics/admin_s3_config.html