topic MODFYING Data in HADOOP from EG. in Administration and Deployment

MODFYING Data in HADOOP from EG.

ankit___gupta — Mon, 06 Feb 2017 11:17:54 GMT

Hi,

i have been trying to update my master data in HADOOP with the transaction data.

So my question is how to modify this master data in hadoop from Enterprise Guide.

LIBNAME hdfs SASHDAT  PATH="/staging"  SERVER=SERVERNODE  INSTALL="/Hadoop/TKGrid";

			data hdfs.MASTER;
				set TRANSACTION;
				modify hdfs.MASTER key=Index;
				set TRANSACTION (drop=ObsCreatedDate);

				if _iorc_= 0 then
					replace;
				else output;
				call missing(of _all_);

			PROC DELETE data=TRANSACTION ;
			run;

It gives the error.

ERROR: The SASHDAT engine is a uni-directional engine. Data flows from the SAS client to the Hadoop Distributed File System. The 
engine cannot be used to fetch data from HDFS.

Then i tried with HPDS2 still the same error please help.

Re: MODFYING Data in HADOOP from EG.

Kurt_Bremser — Mon, 06 Feb 2017 09:41:30 GMT

Your code defines library "hdfs", but the data step uses libraries "staging" and work.

And it is what the message says, SASHDAT can only be used to write data to HDFS, not to read from there. At least in the way you tried.

Did you take your proc hpds2 example from http://support.sas.com/documentation/cdl/en/inmsref/67213/HTML/default/viewer.htm#p0kn1b8a7yt44fn1qwp8w1b8l92w.htm (example 6)?

Please post the complete log, including the libname(s) and data/proc steps.

Re: MODFYING Data in HADOOP from EG.

ankit___gupta — Mon, 06 Feb 2017 11:16:54 GMT

That was a typing error i'll just correct it.

and yes i have tried the hpds2 technique.

Re: MODFYING Data in HADOOP from EG.

LinusH — Tue, 07 Feb 2017 07:25:47 GMT

SASHDAT is mainly for LASR disk storage. What kind if data do you have and why do you need to use modify on it specifically using this engine? The engine is not optimal for DW data.

Re: MODFYING Data in HADOOP from EG.

ankit___gupta — Tue, 07 Feb 2017 09:37:07 GMT

The other engines like SPDE or HADOOP is not licensed to us thats why i was trying to figure all the other possible engine to edit the data.

Re: MODFYING Data in HADOOP from EG.

LinusH — Tue, 07 Feb 2017 14:17:59 GMT

SPDE is included in the Base SAS license, at least in the stand alone one. Not sure if you have "only" Visual Analytics embedded Base, if SPDE is supported. Have you verified this with SAS?

But, don't use a tool that it isn't built for. Do you really need this data to reside in Hadoop?