topic Re: Unable to run Hadoop command in SAS Data Management

Unable to run Hadoop command

vivshri — Sun, 08 Feb 2015 14:15:11 GMT

Hi,

I am trying to setup hadoop connection with SAS 9.3.

filename cfg "C:/LocationOfMyHadoopConfiguration/hadoopconfig.cfg" ;

filename foo hadoop '/tmp/file1.txt' cfg=cfg user='hadoop' pass='mypassword';

data _null_;

file foo;

put "Hadoop Hadoop";

run;

ERROR: java.lang.ClassNotFoundException: com.dataflux.hadoop.DFConfiguration

ERROR: at java.net.URLClassLoader$1.run(Unknown Source)

ERROR: at java.security.AccessController.doPrivileged(Native Method)

ERROR: at java.net.URLClassLoader.findClass(Unknown Source)

ERROR: at java.lang.ClassLoader.loadClass(Unknown Source)

1. I have setup SAS_HADOOP_JAR_PATH in the environment variables so missing environment variable error is no longer coming.

2. I have downloaded all the hadoop jar files in SAS_HADOOP_JAR_PATH ( The jar files are for Hadoop 2.4.0 )

3. The picklist command "proc javainfo picklist 'hadoopbasics/hadoopwrapr.txt'; run;" showing

file:/C:/Program%20Files/SASHome/SASVersionedJarRepository/eclipse/plugins/dfx-hadoop-1.0_301000.

0.0.20121114190000_d3inhd31/dfx-hadoop-1.0.jar

file:/C:/Program%20Files/SASHome/SASVersionedJarRepository/eclipse/plugins/dfx-pig-1.0_301000.0.0

.20121114190000_d3inhd31/dfx-pig-1.0.jar

It seems like I have to specify required jar files in the picklist. is that correct? How to do that?

Appreciate any reply.

Re: Unable to run Hadoop command

LinusH — Mon, 09 Feb 2015 07:14:54 GMT

What Hadoop distribution/version do you use?

Also, consider to at least test in 9.4, since I believe SAS is putting quite some developing effort for Hadoop support.

Perhaps at track to SAS Tech support is a better choice fort his kind of detailed troubleshooting.

Re: Unable to run Hadoop command

vivshri — Wed, 11 Feb 2015 10:46:39 GMT

Thanks Linus, Just to update for future readers, the error is resolved. It seems the SAS_HADOOP_JAR_PATH was not readable. After discussing with SAS support I put that in all user accessible area in the Windows machine and I was able to run hadoop commands from SAS 9.3.

Re: Unable to run Hadoop command

AnnaBrown — Wed, 11 Feb 2015 17:15:43 GMT

Great to hear the error is resolved, vivshri.

Thanks for reporting back on the thread.