Hi,
I am trying to setup hadoop connection with SAS 9.3.
filename cfg "C:/LocationOfMyHadoopConfiguration/hadoopconfig.cfg" ;
filename foo hadoop '/tmp/file1.txt' cfg=cfg user='hadoop' pass='mypassword';
data _null_;
file foo;
put "Hadoop Hadoop";
run;
ERROR: java.lang.ClassNotFoundException: com.dataflux.hadoop.DFConfiguration
ERROR: at java.net.URLClassLoader$1.run(Unknown Source)
ERROR: at java.security.AccessController.doPrivileged(Native Method)
ERROR: at java.net.URLClassLoader.findClass(Unknown Source)
ERROR: at java.lang.ClassLoader.loadClass(Unknown Source)
ERROR: at java.lang.ClassLoader.loadClass(Unknown Source)
1. I have setup SAS_HADOOP_JAR_PATH in the environment variables so missing environment variable error is no longer coming.
2. I have downloaded all the hadoop jar files in SAS_HADOOP_JAR_PATH ( The jar files are for Hadoop 2.4.0 )
3. The picklist command "proc javainfo picklist 'hadoopbasics/hadoopwrapr.txt'; run;" showing
file:/C:/Program%20Files/SASHome/SASVersionedJarRepository/eclipse/plugins/dfx-hadoop-1.0_301000.
0.0.20121114190000_d3inhd31/dfx-hadoop-1.0.jar
file:/C:/Program%20Files/SASHome/SASVersionedJarRepository/eclipse/plugins/dfx-pig-1.0_301000.0.0
.20121114190000_d3inhd31/dfx-pig-1.0.jar
It seems like I have to specify required jar files in the picklist. is that correct? How to do that?
Appreciate any reply.
What Hadoop distribution/version do you use?
Also, consider to at least test in 9.4, since I believe SAS is putting quite some developing effort for Hadoop support.
Perhaps at track to SAS Tech support is a better choice fort his kind of detailed troubleshooting.
Thanks Linus, Just to update for future readers, the error is resolved. It seems the SAS_HADOOP_JAR_PATH was not readable. After discussing with SAS support I put that in all user accessible area in the Windows machine and I was able to run hadoop commands from SAS 9.3.
Great to hear the error is resolved, vivshri.
Thanks for reporting back on the thread.
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.
Find more tutorials on the SAS Users YouTube channel.