Network Timned Out Error in SAS Dataflux

[Shakti_Sourav]

Hi Team,

I am getting Network Timed Error in DM Server (SAS Dataflux). I have attached error log and  Screenshot for your references.please suggest me how to resolve this issue.

[SASKiwi]

This is a SAS data connection to Oracle timeout problem. How long did your job run before timing out? If it is for a long time then this is probably caused by an Oracle server setting or possibly a network firewall rule setting. I suggest you contact an Oracle DBA in your organisation to find out if there is an Oracle server setting causing this. If not, then consult your IT server administrators regarding firewall rule settings.

EDIT: Your job log indicates the Oracle query ran for 2.5 hours which isn't a really long time. Same advice as above applies.

[Shakti_Sourav]

Hi SASKiwi,
I hope you are doing well !!
As per your suggestion, I am checking with it. And this job taking 13 to 14 hours to run.
Start time - 08th July 2022 11:57 AM
End Time - 09th July 2022 02:28 AM

[SASKiwi]

Yes, good thanks as I hope you are. As you have found, DataFlux jobs are quite resource intensive 🙂. We run our DataFlux jobs in parallel to speed them up. One of my jobs took 12 or 13 hours to run. It now runs in 4.5 hours by running three of them in parallel.

You could also consider running the Oracle query in a separate job, just in case it is the following processing that is slowing it down.

[Shakti_Sourav]

As per your suggestion, we also ran the jobs in parallel. but same error I got.

I have attcahed screenshot. please check it, and let me know, is there any solution to jobs run in parallel or it is correct ???

[Shakti_Sourav]

How to run it parallel. I have used fork node and created separate jobs. But it's taking 28 hours to execute contains 7crores of data. And also got network timed out error when reading the data in another job. Please suggest how to do it?

[SASKiwi]

@Shakti_Sourav  - If you want to try parallel processing then I suggest you do a simple test. Take two copies of your single stream job then add a filter to the start of each one so they read a different set of data. Then run the two jobs manually at the same time and see if this speeds up the processing. Doing a simple test first will show you if parallel processing is going to help.

The way I parallel process with DataFlux is very different to the way you are trying it unfortunately. I'm using SAS programs and a SAS product called SAS/CONNECT to call DataFlux jobs to run them in parallel. I suspect you are not familiar with the SAS language and probably don't have SAS/CONNECT so I can't just show you how I do it. I believe it is possible to parallel process just using DataFlux, but to figure it out I'd have to spend a lot of time experimenting, which is time I don't have. I recommend you open a SAS Tech Support track to solve this.

[SASKiwi]

@Shakti_Sourav  - From what I understand, to parallel process only in DataFlux, you create a single Data Job containing the tasks you want to run in parallel. Then you create a Process Job that will then run the Data Job in parallel, using the Fork or Parallel Iterator utilities. I'm not familiar with the details for setting this up so as I already mentioned I suggest you get help from SAS Tech Support.