BookmarkSubscribeRSS Feed
alandool
Quartz | Level 8

Hi ALL,

 

Getting acquainted with SPDE.  I have data sitting in a Hadoop cluster.  I have no problem reading into our SAS Grid until I use ACCELWHERE=YES in the libname as shown below.  Then I get a "generic error."  I am attempting that data subsetting be performed in the Hadoop cluster.  Hopefully, it will bring the data back much quicker.  

 

I can remove the ACCELWHERE=YES in the libname and it will run, but it runs for much too long a period although it eventually completes.  Code below is just a snippet.  (SAS Enterprise Guide version: 7.1.  Platform version: 9.4.3.0) 

 

Appreciate any help or advice . . .

 

Alan

 

libname hdp_spd spde '/hdfs/vs1/rptg/phi' hdfshost=default PARALLELWRITE=YES PARALLELREAD=YES ACCELWHERE=YES ;

 

code ...

from hdp_spd.ltv_201702_copy
where INCRD_YR_MN between &bgn_exp and &end_exp ;

ERROR: Generic error.

2 REPLIES 2
LouGalway_sas
SAS Employee

Hello,

 

Have you seen this documentation?

http://support.sas.com/documentation/cdl/en/spdsug/67358/HTML/default/viewer.htm#p0u8zw2umzaz5zn17ut...

 

Specifically, does your SPD Server have an enabled WHERE?

 

The ACCELWHERE= table option enables a user to override the default state of an undeclared WHERE processing optimization (HADOOPACCELWH=) setting. When SPD Server is using a Hadoop environment, and the SPD Server WHERE processing optimization feature state is not defined in the SPD Server parameter file, the default configuration reverts to NO WHERE processing optimization. You can override an undeclared HADOOPACCELWH= setting by issuing a statement that contains the ACCELWHERE= table option set to YES.
If your SPD Server administrator has enabled WHERE processing optimization via the HADOOPACCELWH= setting, you can use the ACCELWHERE= table option to override and suppress the Hadoop WHERE processing optimization. To override and suppress a HADOOPACCELWH=YES setting, issue a statement that contains the ACCELWHERE= table option set to NO.
 
alandool
Quartz | Level 8

Hi LouGalway_sas,

 

Thanks for input.  Not sure of all our setup details, but we aren't using the SPD Server.

 

Alan

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to connect to databases in SAS Viya

Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 2 replies
  • 843 views
  • 0 likes
  • 2 in conversation