topic Exporting SAS dataset Into Hadoop in SAS Procedures

Exporting SAS dataset Into Hadoop

DrBigAl — Mon, 10 Sep 2018 19:37:14 GMT

I have an issue with exporting a SAS dataset into an Hadoop file location. All of the documentation that I have seen is exporting a file from local computer that is already in txt format.

The file I am trying to export to Hadoop is in my 'work' library and the hdfs path is /Data/direct/abs/

libname out hadoop "/Data/direct/abs/" dir user="&user";

proc sql;
create table out.results_set (COLUMN_DELIMITER = '|') as
select * from work.custlist;
quit;

Re: Exporting SAS dataset Into Hadoop

LinusH — Mon, 10 Sep 2018 19:12:44 GMT

If you want to copy data and store it as a table in Hive you need output to a SAS/ACCESS libref for Hadoop/Impala.
Filename is for files, if that's what you want use a data step with a FILE statement, or PROC EXPORT.

Re: Exporting SAS dataset Into Hadoop

DrBigAl — Mon, 10 Sep 2018 19:39:59 GMT

Thank you. I changed to libname and it is still now working. Says "Engine does not accept a physical name". Could it be my path that it is not accepting?

Re: Exporting SAS dataset Into Hadoop

LinusH — Tue, 11 Sep 2018 06:55:48 GMT

With SAS/ACCESS engines, you usually do not refer to physical (file system) paths. For Hive, you need torefer to a Hive data base.

I think I could be wise ti visit SAS documentation page for SAS/ACCESS to Hadoop, so you can pick up the correct syntax and per-requisites.