BookmarkSubscribeRSS Feed
alandool
Quartz | Level 8

Hi ALL,

 

Getting acquainted with SPDE.  I have data sitting in a Hadoop cluster.  I have no problem reading into our SAS Grid until I use ACCELWHERE=YES in the libname as shown below.  Then I get a "generic error."  I am attempting that data subsetting be performed in the Hadoop cluster.  Hopefully, it will bring the data back much quicker.  

 

I can remove the ACCELWHERE=YES in the libname and it will run, but it runs for much too long a period although it eventually completes.  Code below is just a snippet.  (SAS Enterprise Guide version: 7.1.  Platform version: 9.4.3.0) 

 

Appreciate any help or advice . . .

 

Alan

 

libname hdp_spd spde '/hdfs/vs1/rptg/phi' hdfshost=default PARALLELWRITE=YES PARALLELREAD=YES ACCELWHERE=YES ;

 

code ...

from hdp_spd.ltv_201702_copy
where INCRD_YR_MN between &bgn_exp and &end_exp ;

ERROR: Generic error.

2 REPLIES 2
LouGalway_sas
SAS Alumni

Hello,

 

Have you seen this documentation?

http://support.sas.com/documentation/cdl/en/spdsug/67358/HTML/default/viewer.htm#p0u8zw2umzaz5zn17ut...

 

Specifically, does your SPD Server have an enabled WHERE?

 

The ACCELWHERE= table option enables a user to override the default state of an undeclared WHERE processing optimization (HADOOPACCELWH=) setting. When SPD Server is using a Hadoop environment, and the SPD Server WHERE processing optimization feature state is not defined in the SPD Server parameter file, the default configuration reverts to NO WHERE processing optimization. You can override an undeclared HADOOPACCELWH= setting by issuing a statement that contains the ACCELWHERE= table option set to YES.
If your SPD Server administrator has enabled WHERE processing optimization via the HADOOPACCELWH= setting, you can use the ACCELWHERE= table option to override and suppress the Hadoop WHERE processing optimization. To override and suppress a HADOOPACCELWH=YES setting, issue a statement that contains the ACCELWHERE= table option set to NO.
 
alandool
Quartz | Level 8

Hi LouGalway_sas,

 

Thanks for input.  Not sure of all our setup details, but we aren't using the SPD Server.

 

Alan

CFC_SAS_Communities_400x225.jpg

Call for content now open!

It's your turn to help shape SAS Innovate 2027. Share your expertise and inspire the SAS community.

Submit your proposal →

How to connect to databases in SAS Viya

Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 2 replies
  • 2151 views
  • 0 likes
  • 2 in conversation