Hi Shawn Will definitely try increasing the branch memory cache size option. Also will change the commit frequency and check. BTW, for our job, we are writing to the DB using an expression where we are using the Expression Engine Language and DSNs to write data.
... View more
Hi Shawn I have put that "Lookup Map..." in front of branch. Its executing a bit faster. Anything apart from that which can be implemented, such as changing any of the advanced properties or the configuration files??
... View more
Hey Shawn Thanks a lot for the reply. Following the branch node, i am just adding a few columns and passing some values to those columns. In the "Add GAR.." expression node, I am adding a column. No filtering over there. In the embedded jobs, I am using a data job to check for a certain pattern using a if else statements. if(ascii(x)==49 or ascii(x)==50 . . . . . . or ascii(x)=57) return true else return false
... View more
Hi there!! I have created a complex job which branches out to around 10 flows. It starts with a SQL read node which is reading data from a SQL Server database. The table from which data is being read contains around 3.2 million records. A filter has been added and around 1.6 million records are being written. After reading, the data is being processed for quality. All these is taking up a lot of time - approximately 20 minutes for reading the data and around 4 hours for processing the data and completing the job. Is there any way that DataFlux can be configured to complete jobs/read and process data at a much faster way? Can any configurations be done in order to fine tune the jobs? I have attached a screenshot to illustrate the time the job is taking.
... View more