no sas data written to Hadoop, only metadata BY USING IMPALA

merahilyana — Thu, 09 Jan 2020 09:47:30 GMT

Here is my coding:

libname IPCSIMPA impala dsn = "impala_hadoop_ipcs" schema="myipcs_cdm_raw";

proc sql;
create table IPCSIMPA.E11
as select * from SASDATA.DATABASE_E1_RSTUDIO;
quit;

here is sas log:

1 ;*';*";*/;quit;run;
2 OPTIONS LS=64 PS=15;
3 OPTIONS PAGENO=MIN;
4 %LET _CLIENTTASKLABEL='Program (5)';
5 %LET _CLIENTPROCESSFLOWNAME='Process Flow';
6 %LET
6 ! _CLIENTPROJECTPATH='C:\Users\ipcs_sasdemo\Documents\M
6 ! y SAS Files\Danial_MyIPCS\Hadoop_trial.egp';
7 %LET
7 ! _CLIENTPROJECTPATHHOST='SASMYIPCSOA01.stats.gov.my';
8 %LET _CLIENTPROJECTNAME='Hadoop_trial.egp';
2 The SAS System
10:00 Thursday, January 9, 2020

9 %LET _SASPROGRAMFILE='';
10 %LET _SASPROGRAMFILEHOST='';
11
12 ODS _ALL_ CLOSE;
13 OPTIONS DEV=PNG;
14 GOPTIONS XPIXELS=0 YPIXELS=0;
15 FILENAME EGRTF TEMP;
16 ODS RTF(ID=EGRTF) FILE=EGRTF
17 ENCODING='utf-8'
18 STYLE=Journal2
19 NOGTITLE
20 NOGFOOTNOTE
3 The SAS System
10:00 Thursday, January 9, 2020

21 KEEPN
22 ;
NOTE: Writing RTF Body file: EGRTF
23 FILENAME EGSR TEMP;
24 ODS tagsets.sasreport13(ID=EGSR) FILE=EGSR
25 STYLE=Journal2
26 STYLESHEET=(URL="file:///C:/Program%20Files/SASHo
26 ! me/SASEnterpriseGuide/7.1/Styles/Journal2.css")
27 NOGTITLE
28 NOGFOOTNOTE
29 GPATH=&sasworklocation
30 ENCODING=UTF8
4 The SAS System
10:00 Thursday, January 9, 2020

31 options(rolap="on")
32 ;
NOTE: Writing TAGSETS.SASREPORT13(EGSR) Body file: EGSR
33 FILENAME EGXLSSX TEMP;
34 ODS EXCEL(ID=EGXLSSX) FILE=EGXLSSX STYLE=Excel
35 OPTIONS (
36 EMBEDDED_TITLES="no" EMBEDDED_FOOTNOTES="no"
36 ! SHEET_INTERVAL="None"
37 );
38
39 GOPTIONS ACCESSIBLE;
40 /* Insert custom code before submitted code here */
5 The SAS System
10:00 Thursday, January 9, 2020

41 LIBNAME MYDATA 'C:\EG DATA';
NOTE: Library MYDATA does not exist.
42
43 libname IPCSIMPA impala dsn = "impala_hadoop_ipcs"
43 ! schema="myipcs_cdm_raw";
NOTE: Libref IPCSIMPA was successfully assigned as follows:
Engine: IMPALA
Physical Name: impala_hadoop_ipcs
44
45 proc sql;
46 create table IPCSIMPA.E15
47 as select * from SASDATA.DATABASE_E1_RSTUDIO;
6 The SAS System
10:00 Thursday, January 9, 2020

NOTE: SAS variable labels, formats, and lengths are not written
to DBMS tables.
NOTE: Table IPCSIMPA.E15 created, with 512 rows and 9 columns.

48 quit;
NOTE: PROCEDURE SQL used (Total process time):
real time 1.16 seconds
cpu time 0.03 seconds

49
50 GOPTIONS NOACCESSIBLE;
7 The SAS System
10:00 Thursday, January 9, 2020

51 %LET _CLIENTTASKLABEL=;
52 %LET _CLIENTPROCESSFLOWNAME=;
53 %LET _CLIENTPROJECTPATH=;
54 %LET _CLIENTPROJECTPATHHOST=;
55 %LET _CLIENTPROJECTNAME=;
56 %LET _SASPROGRAMFILE=;
57 %LET _SASPROGRAMFILEHOST=;
58
59 ;*';*";*/;quit;run;
60 ODS _ALL_ CLOSE;
NOTE: Writing EXCEL(EGXLSSX) file:
C:\Users\IPCS_S~1\AppData\Local\Temp\SAS Temporary
8 The SAS System
10:00 Thursday, January 9, 2020

Files\_TD9736_SASMYIPCSOA01_\#LN01219
61
62
63 QUIT; RUN;

the picture is output.

what must i do??

Re: no sas data written to Hadoop, only metadata BY USING IMPALA

Kalind_Patel — Thu, 09 Jan 2020 05:55:15 GMT

Few things you can try to check the root cause:

1. Execute INVALIDATE METADATA statement in your Impala SQL Clients or using Proc SQL Explicit passthrough, sometimes you've to perform this step to get a change reflected in your session, for more details you can reference:
https://impala.apache.org/docs/build/html/topics/impala_invalidate_metadata.html

2. Check the table stats using Proc Contents

3. Last thing is enable the DBMS specific logging,

for example you can run this statement to get a detailed log about database activity:

options sastrace="d,d,d,d";

Re: no sas data written to Hadoop, only metadata BY USING IMPALA

merahilyana — Thu, 09 Jan 2020 09:50:55 GMT

ALL LIST YOU GIVE IS NOT HAVE PROBLEM AND I GET SOLVE MY PROBLEM. I HAVE ONE PROBLEM DATA APPEAR BUT NOT FULL ROW OF DATA APPEAR..

FOR EXAMPLE:
ORIGINAL ROW DATA HAVE 512
BUT OUTPUT IN IMPALA ROW DATA ONLY 176

WHY??

topic Re: no sas data written to Hadoop, only metadata BY USING IMPALA in New SAS User

no sas data written to Hadoop, only metadata BY USING IMPALA

Re: no sas data written to Hadoop, only metadata BY USING IMPALA

Re: no sas data written to Hadoop, only metadata BY USING IMPALA