topic Re: Split the data in Dataflux in SAS Data Management

Split the data in Dataflux

Shakti_Sourav — Thu, 26 May 2022 05:56:30 GMT

Dear Team,

I want to split my DataFlux data into chunks, For example, How to split the 60 million rows into 4,15 million row tables in Data Management Studio ?

Thank You

Shakti

Re: Split the data in Dataflux

ErinW — Sat, 11 Jun 2022 01:39:11 GMT

Hey Shakti!

Have you tried using the Data Validation node? You can use that to filter your data. For example, you could filter on an expression like Profit > 1000. Let me know if that's the sort of thing you're trying to do.

Re: Split the data in Dataflux

VincentRejany — Mon, 13 Jun 2022 06:56:22 GMT

It depends if the split should be done randomly or according to the order rows are read. If this is the later, then add a sequencer node for numbering each row, and next an expression with something like

integer mygroup

integer groups

groups = 4

mygroup = mysequencer % groups

Re: Split the data in Dataflux

RacheLGomez123 — Thu, 03 Nov 2022 08:45:30 GMT

Data splitting is when data is divided into two or more subsets. Typically, with a two-part split, one part is used to evaluate or test the data and the other to train the model.

Data splitting is an important aspect of data science, particularly for creating models based on data. This technique helps ensure the creation of data models and processes that use data models -- such as machine learning -- are accurate.

This may help you,

Rachel Gomez