Campaign performance - Best design practice

[nktech]

Hello Team, 

I am looking for some help to take the best approaches during designing the campaign in Marketing Automation module.

Currently, I can see there are a lot of temporary MA tables are being created under the SASMA folder on compute server which is taking approx 300 GB of space. And campaigns are being failed due to lack of space.

If I understand correctly that temporary tables are created when we used the cell node after select node, is there any document or guidelines which can be followed to create the campaign in the right way and avoid these kinds of the issue at least on functional perspective?

Thanks.

Neeraj

[JamesAnderson]

Hi Neeraj,

Campaigns will always create MATables. These are not only temporary but also represent the previous results of the campaign (i.e. when you open a campaign and it has counts on the node, those counts come from the saved MA tables). Rather than trying to avoid or minimize MA Tables I would focus on 3 things: 

1. Increasing the storage space (300GB is not much)
2. Compress the directory
3. Removing old MATables that are no longer used.

For No3 you can safely delete these from the file system (as long as the campaign isnt running). The easiest way to do this is in a period of low activity (small hours of the morning) and deleting anything older than X months.

Regards

James

[ravindrrapaal]

Hello Neeraj, I agree with James on the mentioned three points. At our end we are having this cleanup activity weekly. I hope you can have batch or shell script depending on OS. And your script can run cleanup on every weekend. Or if this is so much impacting that you can run other campaigns after any specific campaign. You can put cleanup script at end of you campaign, so post completion of the campaign it can clean the data for the run.
Regards,
Ravindrra Paal

[nktech]

Thanks James and Ravinder for your feedback.

In my case, we do have the script which are removing the tables older than 30 days. I am going to reduce the date to 15 days.
If I understand correctly regarding the deletion of matables:

1) these can be deleted by some script as per the OS level
2) after completion of manual or scheduled execution, the matables get removed automatically...correct?
3) if the point #2 is not true then it means the MAtables will remain in the file system until or unless we manually remove them...correct?

In my case, one single campaign took approx 200 GB space and there are many campaigns also running in the system simultaneously, so that is why I asked if there is any guide to explore with users to design the campaign more accurately.

James, regarding compressing the directory...do you mean by compressing/zipping the MA tables, if that is true, wouldn’t this will impact the counts or the execution of the campaign if the associated MAtables are compressed on the file system.

Once again..thanks for the input.

[JamesAnderson]

Some MATables are only used in the session the campaign is executing, and these are deleted after processing is complete. Other MATables represent the counts for a saved campaign, and these remain after the processing is complete. You will always have some MATables left in the file system after a campaign complete.

Regarding compression - I'm referring to filesystem compression not Zip - depending on your operating system it may or may not be available - check with your server admins.

One other thing to avoid is Select Node -> Cell Node -> Select Node. The cell node in the middle serves no purpose and just creates an addition MATable. Also check if the option " Produce counts for all nodes when executing " is turned on - this will increase the amount of space used by MATables.

Can you share a Campaign Diagram PDF of the big campaign ?

[nktech]

Thanks Jim. Please find the attached PDF file.

[JamesAnderson]

Hi,
I cant see a lot wrong with this diagram. There are a few places where there is an extra cell node that isn't required but only a few and that shouldn't cause too much additional overhead. Is it possible that the space is being used by your export files ?
Regards
James