SAS Data Integration Studio, DataFlux Data Management Studio, SAS/ACCESS, SAS Data Loader for Hadoop and others

getting error when using SPDE ACCELWHERE=YES in libname

Reply
Contributor
Posts: 24

getting error when using SPDE ACCELWHERE=YES in libname

Hi ALL,

 

Getting acquainted with SPDE.  I have data sitting in a Hadoop cluster.  I have no problem reading into our SAS Grid until I use ACCELWHERE=YES in the libname as shown below.  Then I get a "generic error."  I am attempting that data subsetting be performed in the Hadoop cluster.  Hopefully, it will bring the data back much quicker.  

 

I can remove the ACCELWHERE=YES in the libname and it will run, but it runs for much too long a period although it eventually completes.  Code below is just a snippet.  (SAS Enterprise Guide version: 7.1.  Platform version: 9.4.3.0) 

 

Appreciate any help or advice . . .

 

Alan

 

libname hdp_spd spde '/hdfs/vs1/rptg/phi' hdfshost=default PARALLELWRITE=YES PARALLELREAD=YES ACCELWHERE=YES ;

 

code ...

from hdp_spd.ltv_201702_copy
where INCRD_YR_MN between &bgn_exp and &end_exp ;

ERROR: Generic error.

SAS Employee
Posts: 3

Re: getting error when using SPDE ACCELWHERE=YES in libname

Hello,

 

Have you seen this documentation?

http://support.sas.com/documentation/cdl/en/spdsug/67358/HTML/default/viewer.htm#p0u8zw2umzaz5zn17ut...

 

Specifically, does your SPD Server have an enabled WHERE?

 

The ACCELWHERE= table option enables a user to override the default state of an undeclared WHERE processing optimization (HADOOPACCELWH=) setting. When SPD Server is using a Hadoop environment, and the SPD Server WHERE processing optimization feature state is not defined in the SPD Server parameter file, the default configuration reverts to NO WHERE processing optimization. You can override an undeclared HADOOPACCELWH= setting by issuing a statement that contains the ACCELWHERE= table option set to YES.
If your SPD Server administrator has enabled WHERE processing optimization via the HADOOPACCELWH= setting, you can use the ACCELWHERE= table option to override and suppress the Hadoop WHERE processing optimization. To override and suppress a HADOOPACCELWH=YES setting, issue a statement that contains the ACCELWHERE= table option set to NO.
 
Contributor
Posts: 24

Re: getting error when using SPDE ACCELWHERE=YES in libname

Hi LouGalway_sas,

 

Thanks for input.  Not sure of all our setup details, but we aren't using the SPD Server.

 

Alan

Ask a Question
Discussion stats
  • 2 replies
  • 218 views
  • 0 likes
  • 2 in conversation