How to run SAS ETL jobs parallely?

Reply
Frequent Contributor
Posts: 101

How to run SAS ETL jobs parallely?

I want to run some SAS ETL sequence jobs in parallel.

Is there any way to achieve this in SAS Data Integrator Studio?

Kindly help me on this.

Frequent Contributor
Posts: 88

Re: How to run SAS ETL jobs parallely?

Posted in reply to Helannivas

Hi Helannivas,

SAS Data Integration Studio uses a set of macros to enable parallel processing. You can enable these macros by doing one of the following:

  • selecting YES in the Enable parallel processing macros option on the Options tab of the properties window for a job.
  • including a Loop transformation in a job.

When you enable the parallel-processing option for a job, macros are generated at the top of the job code with comments. These macros enable you to create your own transformations or code in order to use parallel processing.

When you include a Loop transformation in a job, the transformation generates the necessary macros to use sequential execution, symmetric multiprocessing (SMP) execution, or execution on a grid computing network.

No special software or metadata is required to enable parallel processing on SMP servers. Grid options can be enabled for a job even when the grid software has not been configured and licensed. However, SAS Data Integration Studio does not generate grid-enabled code for the job in this case. It generates code that is appropriate for SMP on the SAS Application Server.

The following table describes the prerequisites that are required to enable parallel processing for SAS Data Integration Studio jobs. For details about these prerequisites, see the appropriate section in the documentation mentioned below.

Prerequisites for Parallel Processing of SAS Data Integration Studio Jobs

 

 

Computers Used for Parallel Processing

 

Requirements

 

SMP machine with one or more processors

 

Specify a SAS®9 Workspace server in the metadata for the default for SAS Data Integration Studio. See the "Specifying Metadata for the Default SAS Application Server" topic in SAS Data Integration Studio Help.

 

Grid computing network

 

Specify an appropriate SAS Metadata Server to get the latest metadata object for a grid server. See the SAS Data Integration Studio chapter in the SAS Intelligence Platform: Desktop Application Administration Guide.

Specify an appropriate SAS®9 Workspace Server in the metadata for the default.

Grid software must be licensed.

Define or add a grid server component to the metadata that points to the grid server installation. The controlling server machine must have both a grid server definition and a SAS Workspace Server definition as a minimum to be able to run your machines in a grid. It is recommended that you also have the SAS Metadata Server component accessible to the server definition where your grid machines are located.

Install Platform Computing software to handle workload management for the grid.

 

Regards,

Tushar J.

Message was edited by: Tushar Jagtap

Ask a Question
Discussion stats
  • 1 reply
  • 405 views
  • 0 likes
  • 2 in conversation