Hello, Among my tasks, I have developed a program to validate SAS dataset in term of structure, then in term of values (constraints). After that, I need to convert my validated data set into an xml file. However, we have external data set of about 500 variables and 35 millions of observations. The conversion is very too long. I have made some test and I found that fragmenting the dataset into smaller data set then converting them permit to reduce the process time, let's say of about 40%. It still remain to long In fact to convert a dataset of 480 variables and 25000 observations, it takes about 3 minutes. It is again to long. I wrote to SAS to ask them if they could improve their xmlv2 engine but without success. Thereafter, I have look for the parallel processing approach, and my preliminary test with other dataset permit me to reduce again the process time of about 50%. So I have elaborate a project which read 4 datasets of various size, and convert those into xml file (serial process). After that, we read the same datasets and convert those into xml file using parallel processing. we observe gains in terms of processing time (-50%). It was suggested to use call execute to correct the error observed in the project. After that, I would like to test the parallel approach with a large fragmented data set to see the gain and if this approach could be useful for our project. I let you know if the bench test project is working. regards,
... View more