Hello,
I have a problem in work SAS HPDM. I running SAS code and see a warning message:
WARNING: The datanode running on cloudera-node2.example.loc did not respond. No file blocks are written on that host. ?
How to debug connection to the datanode cloudera-node2.example.loc ?
I use CLOUDERA HADOOP 5.11, SAS 9.4, TKGrid, SAS Plug-ins for Hadoop package (SASHDAT), SAS Embedded Process for Hadoop, RHEL 6.8
Hadoop cluster has 4 nodes.
7 proc hpatest;
8 performance nodes=all details;
9 run;
NOTE: The HPATEST procedure is executing in the distributed computing environment with 2 worker nodes. The SAS System 18:31 Tuesday, October 3, 2017 1
Performance Information
Host Node cloudera-node1
Execution Mode Distributed
Number of Compute Nodes 2
NOTE: The PROCEDURE HPATEST printed page 1.
NOTE: PROCEDURE HPATEST used (Total process time):
real time 2.01 seconds
cpu time 0.06 seconds
10
11 libname hdatLib sashdat path="/data/hps" verbose=yes;
NOTE: Libref HDATLIB was successfully assigned as follows:
Engine: SASHDAT
Physical Name: Directory '/data/hps' of HDFS cluster on host 'cloudera-node1'
12
13 data simData;
14 array _a{8} _temporary_ (0,0,0,1,0,1,1,1);
15 array _b{8} _temporary_ (0,0,1,0,1,0,1,1);
16 array _c{8} _temporary_ (0,1,0,0,1,1,0,1);
17 do obsno=1 to 10000000;
18 x = rantbl(1,0.28,0.18,0.14,0.14,0.03,0.09,0.08,0.06);
19 a1 = _a{x};
20 b1 = _b{x};
21 c1 = _c{x};
22 x1 = int(ranuni(1)*400);
23 x2 = 52 + ranuni(1)*38;
2 The SAS System 18:31 Tuesday, October 3, 2017
24 x3 = ranuni(1)*12;
25 lp = 6. -0.015*(1-a1) + 0.7*(1-b1) + 0.6*(1-c1) + 0.02*x1 -0.05*x2 - 0.1*x3;
26 y1 = ranbin(1,1,(1/(1+exp(lp))));
27 output;
28 end;
29 drop x lp;
30 run;
NOTE: The data set WORK.SIMDATA has 10000000 observations and 8 variables.
NOTE: DATA statement used (Total process time):
real time 3.36 seconds
cpu time 3.36 seconds
31
32 data hdatLib.simData;
33 set simData;
34 run;
WARNING: The datanode running on cloudera-node2.example.loc did not respond. No file blocks are written on that host.
NOTE: There were 10000000 observations read from the data set WORK.SIMDATA.
NOTE: The data set /data/hps/simdata has 10000000 observations and 8 variables.
NOTE: DATA statement used (Total process time):
real time 15.49 seconds
cpu time 1.79 seconds
