We are trying to write data from SAS to HDFS ,SAS Enterprise Guide application able to write data into HDFS and logs shows writing data into HDFS ,but some reason we are not able to see data in below hadoop cluster and its shows 0 file size ,please provide me suggestion on this issue.
---code -------------------------------------------
libname hdfslib hadoop server='abcd.com'
HDFS_TEMPDIR='/user/rhive/test'
HDFS_DATADIR='/user/rhive/lib'
HDFS_METADIR='/user/rhive/testabc';
/* Copy SASHELP.CARS over to HDFS */
data hdfslib.carsdata7;
set sashelp.cars;
run;
libname hdfslib clear;
-------------------------------------------------
logfile deatils below :
NOTE: Writing HTML(EGHTML) Body file: EGHTML
21
22 GOPTIONS ACCESSIBLE;
23 libname hdfslib hadoop server='abcd.com'
24 HDFS_TEMPDIR='/user/rhive/test'
25 HDFS_DATADIR='/user/rhive/lib'
26 HDFS_METADIR='/user/rhive/testabc';
NOTE: Libref HDFSLIB was successfully assigned as follows:
Engine: HADOOP
Physical Name: /user/rhive/testabc
27
28 /* Copy SASHELP.CARS over to HDFS */
29 data hdfslib.carsdata7;
30 set sashelp.cars;
31 run;
NOTE: SAS variable labels, formats, and lengths are not written to DBMS tables.
NOTE: There were 428 observations read from the data set SASHELP.CARS.
NOTE: The data set HDFSLIB.CARSDATA7 has 428 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 2.68 seconds
user cpu time 0.03 seconds
system cpu time 0.02 seconds
memory 1916.43k
OS Memory 18404.00k
Timestamp 08/04/2017 04:47:31 PM
Step Count 2 Switch Count 33
Page Faults 0
Page Reclaims 1856
Page Swaps 0
Voluntary Context Switches 3732
Involuntary Context Switches 0
Block Input Operations 0
Block Output Operations 0
32
33 libname hdfslib clear;
2 The SAS System 16:46 Friday, August 4, 2017
NOTE: Libref HDFSLIB has been deassigned.
34
35
36
37
38 GOPTIONS NOACCESSIBLE;
39 %LET _CLIENTTASKLABEL=;
40 %LET _CLIENTPROJECTPATH=;
41 %LET _CLIENTPROJECTNAME=;
42 %LET _SASPROGRAMFILE=;
43
44 ;*';*";*/;quit;run;
45 ODS _ALL_ CLOSE;
46
47
48 QUIT; RUN;
49
-------------------
hadoop cluster data ,its showing 0 file size
-rw-r--r-- 3 b_sas hdmi-technology 0 2017-08-04 16:10 /user/rhive/testabc/carsdata7.sashdmd
Hi @a_SAS
This is most likely a configuration problem with the XML files. Current versions of SAS can be configured using the hadooptracer.py Python tool. I have seen it quite often with the Cloudera Quickstart and Hortonworks Sandbox, too.
The configuration guides for SAS/ACCESS to Hadoop can be found here:
This SAS Communities thread may help:
Best wishes,
Jeff
Thank you for reply ,finally i figure out this issue.
Just am sharing ,this is the below path removed from Yarn-site.xml file.
After removing this below line,its allow to write SAS data into HDFS
Yarn-site.xml
<xi:include href="rmha-site.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
Feels like something worth contacting SAS Tech Support.
Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.
Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.
What’s the difference between SAS Enterprise Guide and SAS Studio? How are they similar? Just ask SAS’ Danny Modlin.
Find more tutorials on the SAS Users YouTube channel.