Desktop productivity for business analysts and programmers

Unable to write data from SAS to HDFS

Reply
New Contributor
Posts: 2

Unable to write data from SAS to HDFS

 

We are trying to write data from SAS to HDFS ,SAS Enterprise Guide application able to write data into HDFS and logs shows writing data into HDFS ,but some reason we are not able to see data in below hadoop cluster and its shows 0 file size  ,please provide me suggestion on this issue.

 

---code -------------------------------------------

libname hdfslib hadoop server='abcd.com'
HDFS_TEMPDIR='/user/rhive/test'
HDFS_DATADIR='/user/rhive/lib'
HDFS_METADIR='/user/rhive/testabc';

/* Copy SASHELP.CARS over to HDFS */
data hdfslib.carsdata7;
set sashelp.cars;
run;

libname hdfslib clear;

-------------------------------------------------

logfile deatils below : 

 

NOTE: Writing HTML(EGHTML) Body file: EGHTML
21
22 GOPTIONS ACCESSIBLE;
23 libname hdfslib hadoop server='abcd.com'
24 HDFS_TEMPDIR='/user/rhive/test'
25 HDFS_DATADIR='/user/rhive/lib'
26 HDFS_METADIR='/user/rhive/testabc';
NOTE: Libref HDFSLIB was successfully assigned as follows:
Engine: HADOOP
Physical Name: /user/rhive/testabc
27
28 /* Copy SASHELP.CARS over to HDFS */
29 data hdfslib.carsdata7;
30 set sashelp.cars;
31 run;

NOTE: SAS variable labels, formats, and lengths are not written to DBMS tables.
NOTE: There were 428 observations read from the data set SASHELP.CARS.
NOTE: The data set HDFSLIB.CARSDATA7 has 428 observations and 15 variables.
NOTE: DATA statement used (Total process time):
real time 2.68 seconds
user cpu time 0.03 seconds
system cpu time 0.02 seconds
memory 1916.43k
OS Memory 18404.00k
Timestamp 08/04/2017 04:47:31 PM
Step Count 2 Switch Count 33
Page Faults 0
Page Reclaims 1856
Page Swaps 0
Voluntary Context Switches 3732
Involuntary Context Switches 0
Block Input Operations 0
Block Output Operations 0

32
33 libname hdfslib clear;
2 The SAS System 16:46 Friday, August 4, 2017

NOTE: Libref HDFSLIB has been deassigned.
34
35
36
37
38 GOPTIONS NOACCESSIBLE;
39 %LET _CLIENTTASKLABEL=;
40 %LET _CLIENTPROJECTPATH=;
41 %LET _CLIENTPROJECTNAME=;
42 %LET _SASPROGRAMFILE=;
43
44 ;*';*";*/;quit;run;
45 ODS _ALL_ CLOSE;
46
47
48 QUIT; RUN;
49

-------------------

hadoop cluster data ,its showing 0 file size


-rw-r--r-- 3 b_sas hdmi-technology 0 2017-08-04 16:10 /user/rhive/testabc/carsdata7.sashdmd

SAS Employee
Posts: 263

Re: Unable to write data from SAS to HDFS

Hi @a_SAS

 

This is most likely a configuration problem with the XML files. Current versions of SAS can be configured using the hadooptracer.py Python tool. I have seen it quite often with the Cloudera Quickstart and Hortonworks Sandbox, too. 

 

The configuration guides for SAS/ACCESS to Hadoop can be found here:

https://support.sas.com/en/documentation/third-party-software-reference/9-4/guides-papers-for-hadoop...

 

This SAS Communities thread may help:

https://communities.sas.com/t5/SAS-Data-Management/No-sasdata-written-to-Hadoop-only-metadata/m-p/15...

 

Best wishes,

Jeff

 

New Contributor
Posts: 2

Re: Unable to write data from SAS to HDFS

Thank you for reply ,finally i figure out this issue.

 

Just am sharing ,this is the below path removed from Yarn-site.xml file.  

 

After removing this below line,its allow to write SAS data into HDFS

 

Yarn-site.xml

 

<xi:include href="rmha-site.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />

Ask a Question
Discussion stats
  • 2 replies
  • 462 views
  • 2 likes
  • 2 in conversation