topic Re: Query regarding bulkload from SAS to Hadoop in SAS Programming

Query regarding bulkload from SAS to Hadoop

india2016 — Mon, 02 May 2022 04:59:23 GMT

Hello Experts,

I need your quick help regarding bulkload option.

Scenario:

I am trying to load data into Hadoop library from SAS work library. Trying with sample 10 records and after 15minutes wait time, 0 records has been loaded to hadoop table..

Tried 3ways with Bulkload=Yes option but getting same output.

1) Data Hdp.abc(bulkload=yes);

set sashelp.cars(obs=10);

run;

2) proc append base=Hdp.abc(bulkload=yes) data=work.cars;

run;

3) created table structre and tried to pust using Insert into statement.

Please suggest the fastet way to load data into Hadoop table

or let me know do I need to do any other changes in config file to work BULKLOAD option.

Re: Query regarding bulkload from SAS to Hadoop

JosvanderVelden — Mon, 02 May 2022 09:00:30 GMT

Have you seen the example from the documentation: https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/acreldb/n0mnrn0q9n41atn194mujpi4zel9.htm

Re: Query regarding bulkload from SAS to Hadoop

Patrick — Mon, 02 May 2022 10:10:22 GMT

Your syntax should work. Do you get any error or warning messages or is the SAS job just hanging?

Is there any SAS process that already loads data into this Hadoop cluster?

Re: Query regarding bulkload from SAS to Hadoop

india2016 — Tue, 03 May 2022 04:24:41 GMT

Hello Patrick,

I am not getting any warning or Error. Job is running as usual with long time and did not load single row into Hadoop table and there is no job which loads data into hadoop cluster.
Trying to load data first time.

Re: Query regarding bulkload from SAS to Hadoop

Patrick — Tue, 03 May 2022 08:47:36 GMT

Given you run a proc append I would expect that your job either fails or data gets loaded.

IF data gets loaded then we could look into performance, if it fails then first why and where it fails.

What does the SAS log tell you for the Proc Append step? Please share the code.

Ideally run your job with following options set before the Proc Append:

option sastrace=',,,d' sastraceloc=saslog nostsuffix;

And not sure and haven't checked if it also applies for Hadoop but also set the following option:

option DBIDIRECTEXEC;

And then share the SAS log for at least the Proc Append step.

Re: Query regarding bulkload from SAS to Hadoop

LinusH — Tue, 03 May 2022 08:57:28 GMT

Have you tried to use

options sastrace=",,,ds" sastraceloc=saslog nostsuffix msglevel=i;

Re: Query regarding bulkload from SAS to Hadoop

SASKiwi — Tue, 03 May 2022 20:19:57 GMT

Does it work without the BULKLOAD option? Have you checked with your SAS administrator regarding any site-specific requirements?

Re: Query regarding bulkload from SAS to Hadoop

india2016 — Wed, 04 May 2022 11:59:01 GMT

Yes LinusH , I have tried these options as well but could not push data into Hadoop table.

Re: Query regarding bulkload from SAS to Hadoop

india2016 — Wed, 04 May 2022 12:03:49 GMT

Hello SAS Kiwi,

I tried without bulkload as well but same result.

and can you please tell me, what site-specific requirements needs to check with Admin?

Sorry I did not get your question.

Re: Query regarding bulkload from SAS to Hadoop

LinusH — Wed, 04 May 2022 12:19:46 GMT

These options doesn't make it work.
What you could hope for is more informatIon in the log (which could be posted here).

Re: Query regarding bulkload from SAS to Hadoop

SASKiwi — Thu, 05 May 2022 04:14:21 GMT

I assume that someone in your organisation has actually tested loading SAS data into Hadoop. Your SAS administrator should know what the coding requirements are to do this successfully - ask them what they are. If they don't know they should know someone who does. Otherwise I suggest you open a track with SAS Tech Support.

Re: Query regarding bulkload from SAS to Hadoop

india2016 — Fri, 06 May 2022 04:19:08 GMT

Thanks SAS Kiwi ,

I have raised ticket with SAS tech support .

THanks for your input.