Split the data in Dataflux

[Shakti_Sourav]

Dear Team,

I want to split my DataFlux data into chunks, For example, How to split the 60 million rows into 4,15 million row tables in Data Management Studio ?

Thank You

Shakti

[ErinW]

Hey Shakti!

Have you tried using the Data Validation node? You can use that to filter your data. For example, you could filter on an expression like Profit > 1000. Let me know if that's the sort of thing you're trying to do.

[VincentRejany]

It depends if the split should be done randomly or according to the order rows are read. If this is the later, then add a sequencer node for numbering each row, and next an expression with something like

integer mygroup

integer groups

groups = 4

mygroup = mysequencer % groups

[RacheLGomez123]

Data splitting is when data is divided into two or more subsets. Typically, with a two-part split, one part is used to evaluate or test the data and the other to train the model.

Data splitting is an important aspect of data science, particularly for creating models based on data. This technique helps ensure the creation of data models and processes that use data models -- such as machine learning -- are accurate.

This may help you,

Rachel Gomez