dear all:
I use jdbc to connect to hive database ,the libname code is
libname dm JDBC url="jdbc:hive2://xx.xx.xx.xx:10000/" classpath="D:\JDBC_Drivers_For_Hive" driverclass="com.cloudera.hive.jdbc.HS2Driver"
user="*******" password="*********" schema="DM" DBMAX_TEXT=1024 PRESERVE_TAB_NAMES=YES PRESERVE_COL_NAMES=YES TRANSCODE_FAIL=WARNING QUOTE_CHAR='"' ;
libname mylib "d:\";
proc sql;
connect using DM;
create table mylib.the_big_datas
select * from connection to DM
(
select * from dm.the_big_dat
);
disconnect from DM;
quit;
I try to select a big data from the hive database which has 1000 columns and 1.8 million rows. Here I got the error log as below:
ERROR: Fetch error: [Cloudera][HiveJDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 30. Background thread ID: 31. Error caught: Java heap space.
ERROR: Close error: java.lang.OutOfMemoryError: Java heap space
what should I do to make the SQL query statements work ?
ANY suggestion is welcome!
Thanks in advance!
Have you tried to do a smilar query from another tool (than SAS)?
I found this, not sure if it's applicable:
Registration is now open for SAS Innovate 2025 , our biggest and most exciting global event of the year! Join us in Orlando, FL, May 6-9.
Sign up by Dec. 31 to get the 2024 rate of just $495.
Register now!
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.
Ready to level-up your skills? Choose your own adventure.