BookmarkSubscribeRSS Feed
☑ This topic is solved. Need further help from the community? Please sign in and ask a new question.
buddha_d
Pyrite | Level 9

I have the following code to fix the Buffer Errors in HADOOP sql:

 

set tez.am.resource.memory.mb=12288;
set hive.tez.container.size=12288;
set tez.runtime.io.sort.mb=1638;
set hive.tez.java.opts=-Xmx8192M;

 

 

I don't know how to use it in SAS code in PROC SQL pass through query. 

I tried different options like 
1.

PROC SQL;

connect to hadoop( );
create table WANT as select * from connection to hadoop
(

set tez.am.resource.memory.mb=12288;
set hive.tez.container.size=12288;
set tez.runtime.io.sort.mb=1638;
set hive.tez.java.opts=-Xmx8192M;

select * 

from datasetname
);

disconnect from hadoop;

quit; 

 

2.

data _null_ ;

set tez.am.resource.memory.mb=12288;
set hive.tez.container.size=12288;
set tez.runtime.io.sort.mb=1638;
set hive.tez.java.opts=-Xmx8192M;

run;

 

I am getting this error:

ERROR: The libraries for both data sets of data set list (.tez-WORK.Xmx8192M) must be the same.
ERROR 211-185: Invalid data set name.

ERROR 22-322: Syntax error, expecting one of the following: a name, a quoted string, (, -, :, ;, CUROBS, END, INDSNAME, KEY,
KEYRESET, KEYS, NOBS, OPEN, POINT, _DATA_, _LAST_, _NULL_.

ERROR 76-322: Syntax error, statement will be ignored.

ERROR 200-322: The symbol is not recognized and will be ignored.

 

Thanks in advance!!!

1 ACCEPTED SOLUTION

Accepted Solutions
yabwon
Onyx | Level 15

1) Show us more of your log

2) Try execute each set in separate statement, and without the semicolon inside, like:

execute
(
set tez.am.resource.memory.mb=12288
) by hadoop;

Bart

_______________
Polish SAS Users Group: www.polsug.com and communities.sas.com/polsug

"SAS Packages: the way to share" at SGF2020 Proceedings (the latest version), GitHub Repository, and YouTube Video.
Hands-on-Workshop: "Share your code with SAS Packages"
"My First SAS Package: A How-To" at SGF2021 Proceedings

SAS Ballot Ideas: one: SPF in SAS, two, and three
SAS Documentation



View solution in original post

4 REPLIES 4
yabwon
Onyx | Level 15

Did you tried execute?

PROC SQL;

connect to hadoop( );

execute
(
set tez.am.resource.memory.mb=12288;
set hive.tez.container.size=12288;
set tez.runtime.io.sort.mb=1638;
set hive.tez.java.opts=-Xmx8192M;
) by hadoop;

/* create table query goes here */

disconnect from hadoop;

quit; 

Doc:

https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/sqlproc/n0tpd3yaqvep53n1g8wahav3hgco.htm

_______________
Polish SAS Users Group: www.polsug.com and communities.sas.com/polsug

"SAS Packages: the way to share" at SGF2020 Proceedings (the latest version), GitHub Repository, and YouTube Video.
Hands-on-Workshop: "Share your code with SAS Packages"
"My First SAS Package: A How-To" at SGF2021 Proceedings

SAS Ballot Ideas: one: SPF in SAS, two, and three
SAS Documentation



buddha_d
Pyrite | Level 9

yabwon,

It didn't work as you suggested. I am getting the following error when I try your code: 

 

ERROR: Prepare error: [Cloudera][HiveJDBCDriver](500051) ERROR processing query/statement. Error Code: 1, SQL state: Error while
compiling statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask, Query: select

yabwon
Onyx | Level 15

1) Show us more of your log

2) Try execute each set in separate statement, and without the semicolon inside, like:

execute
(
set tez.am.resource.memory.mb=12288
) by hadoop;

Bart

_______________
Polish SAS Users Group: www.polsug.com and communities.sas.com/polsug

"SAS Packages: the way to share" at SGF2020 Proceedings (the latest version), GitHub Repository, and YouTube Video.
Hands-on-Workshop: "Share your code with SAS Packages"
"My First SAS Package: A How-To" at SGF2021 Proceedings

SAS Ballot Ideas: one: SPF in SAS, two, and three
SAS Documentation



buddha_d
Pyrite | Level 9
This works for me. Giving each statement is the way to go. Thanks a ton yabwon !!!

You Rock. I appreciate it.

hackathon24-white-horiz.png

The 2025 SAS Hackathon Kicks Off on June 11!

Watch the live Hackathon Kickoff to get all the essential information about the SAS Hackathon—including how to join, how to participate, and expert tips for success.

YouTube LinkedIn

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

SAS Training: Just a Click Away

 Ready to level-up your skills? Choose your own adventure.

Browse our catalog!

Discussion stats
  • 4 replies
  • 874 views
  • 1 like
  • 2 in conversation