topic Re: Modify dataset in HADOOP ? in SAS Data Management

Modify dataset in HADOOP ?

ankit___gupta — Mon, 30 Jan 2017 09:34:53 GMT

Hi,

I am trying to do the following.

--> Create a SPDE library in Hadoop.

--> Send data to this library.

--> to create index on the dataset.

--> And be able to modify the dataset.

I don't know how to create SPDE library in SAS.

for the rest i have excuted the following.

LIBNAME STAGING SASHDAT  PATH="/staging"  SERVER=SASSERVERNODE  INSTALL="/Hadoop/TKGrid" ;
LIBNAME webma ODBC  DATAsrc=DSNAME USER=username  PASSWORD="{SAS002}4B4F181326F9896F4177761F1B986997140F7C995AE23782" ;

PROC SQL INOBS=10;
	create table STAGING.Big as
		Select * from webma.tblmaclaim;
quit;


proc sql;
    create index inde on STAGING.Big(ClmID, Clmtypeid);
quit;

DATA STAGING.Big;
	modify STAGING.Big;
	if ClmID=100008 then
		ClmEPD = 100;
run;

It gives the folllowing logs.

22         
23         GOPTIONS ACCESSIBLE;
24         proc sql;
25             create index inde on STAGING.Big(ClmID, Clmtypeid);
ERROR: Update access is not supported for file STAGING.BIG.DATA.
NOTE: PROC SQL set option NOEXEC and will continue to check the syntax of statements.
26         quit;

I think the problem might be the SASHDAT engine.

Please help.

-Thanks in advance.

Re: Modify dataset in HADOOP ?

LinusH — Mon, 30 Jan 2017 10:23:53 GMT

You are not using SPDE:

http://support.sas.com/documentation/cdl/en/engspdehdfsug/69725/HTML/default/viewer.htm#n1s4fhx0fko8zkn1fiinudodmmai.htm

Re: Modify dataset in HADOOP ?

ankit___gupta — Mon, 30 Jan 2017 13:37:55 GMT

Hi LinusH,

I do not have SPDE, as it is not there in our license. I'm stuck.

Is it possblie to do it with SASHDAT itself ?

Re: Modify dataset in HADOOP ?

maggiem_sas — Thu, 02 Feb 2017 16:02:56 GMT

What SAS version and product? You might have SPD Engine.

Have you tried submitting a LIBNAME with SPDE? See the examples here:

http://support.sas.com/documentation/cdl/en/engspdehdfsug/69725/HTML/default/viewer.htm#titlepage.htm

in the chapter "How to Use Hadoop Data Storage."

Re: Modify dataset in HADOOP ?

BrunoMueller — Thu, 02 Feb 2017 16:46:13 GMT

You can make use of the SPDE engine but store the data in HDFS.

An example LIBNAME would look like this

libname spd_hdfs spde '/user/yourUser' hdfshost=default ;

Check the doc for more options.

Bruno