Modify dataset in HADOOP ?

[ankit___gupta]

Hi,

I am trying to do the following.

                                --> Create a SPDE library in Hadoop.

                                --> Send data to this library.

                                --> to create index on the dataset.

                                --> And be able to modify the dataset.

I don't know how to create SPDE library in SAS.

for the rest i have excuted the following.

LIBNAME STAGING SASHDAT  PATH="/staging"  SERVER=SASSERVERNODE  INSTALL="/Hadoop/TKGrid" ;
LIBNAME webma ODBC  DATAsrc=DSNAME USER=username  PASSWORD="{SAS002}4B4F181326F9896F4177761F1B986997140F7C995AE23782" ;

PROC SQL INOBS=10;
	create table STAGING.Big as
		Select * from webma.tblmaclaim;
quit;


proc sql;
    create index inde on STAGING.Big(ClmID, Clmtypeid);
quit;

DATA STAGING.Big;
	modify STAGING.Big;
	if ClmID=100008 then
		ClmEPD = 100;
run;

It gives the folllowing logs.

22         
23         GOPTIONS ACCESSIBLE;
24         proc sql;
25             create index inde on STAGING.Big(ClmID, Clmtypeid);
ERROR: Update access is not supported for file STAGING.BIG.DATA.
NOTE: PROC SQL set option NOEXEC and will continue to check the syntax of statements.
26         quit;

I think the problem might be the SASHDAT engine.

Please help.

-Thanks in advance.

[LinusH]

You are not using SPDE:

http://support.sas.com/documentation/cdl/en/engspdehdfsug/69725/HTML/default/viewer.htm#n1s4fhx0fko8...

[ankit___gupta]

Hi LinusH,

I do not have SPDE, as it is not there in our license. I'm stuck.

Is it possblie to do it with SASHDAT itself ?

[maggiem_sas]

What SAS version and product? You might have SPD Engine.

Have you tried submitting a LIBNAME with SPDE? See the examples here:

http://support.sas.com/documentation/cdl/en/engspdehdfsug/69725/HTML/default/viewer.htm#titlepage.ht...

in the chapter "How to Use Hadoop Data Storage."

[BrunoMueller]

Hi

You can make use of the SPDE engine but store the data in HDFS.

An example LIBNAME would look like this

libname spd_hdfs spde '/user/yourUser' hdfshost=default ;

Check the doc for more options.

Bruno