We’re smarter together. Learn from this collection of community knowledge and add your expertise.

How to Limit the Number of Input Rows in a DataFlux Data Management Studio Data Job

by SAS Employee MKQueen on ‎03-13-2017 03:35 PM - edited on ‎03-13-2017 04:23 PM by Community Manager (615 Views)

Sometimes you may want to limit the number of rows you are reading in your DataFlux Data Management Studio Data Job. This is especially useful when trying to design and test your data jobs for large input data sets.  

 

 

There is an advanced property called MAX_OUTPUT_ROWS that can be used to limit the number of rows read. This property is available on all the Data Input nodes in a Data Management Studio data job. To view the advanced properties for a node, select the node in the data flow and then select 0_Action_button.png > Advanced Properties from the toolbar. It is a best practice to set the value of MAX_OUTPUT_ROWS property using a macro variable as shown below. 1_MAX_OUTPUT_ROWS.png

Figure 1: MAX_OUTPUT_ROWS Advanced Property

 

 

Using a macro variable for this setting means you can easily change the number of rows you are reading to increase or decrease the number of records you are working with in your data job. In the example below, the macro variable has been set to 10; therefore, only 10 rows will be read in from the data source.

2_NUM_ROWS_TO_READ_Macro_Setting.png 

Figure 2:  NUM_ROWS_TO_READ Macro Variable Setting

 

 

As expected the preview of the node below shows only the first 10 records for the data source. 3_MAX_OUTPUT_ROWS_Preview.png Figure 3:  MAX_OUTPUT_ROWS Data Job Preview

 

 

Setting the macro variable value to 0 means that all rows will be read in from the input data source. Typically, in your macro variable settings on your DataFlux Data Management Server you will have the MAX_OUTPUT_ROWS macro variable name you used in your data jobs set to 0, so all rows will be read in from you input data sources when running data jobs on the server.    

 

 

For more information on Advanced properties for nodes in DataFlux Data Management Studio, refer to the topic “Advanced Properties” in the DataFlux Data Management Studio 2.7: User’s Guide.    

 

Contributors
Your turn
Sign In!

Want to write an article? Sign in with your profile.