Hi All, I am trying to identify the best method for handling 1 or more txt files in Data Management Studio 2.7, being received from different sources but in the same format and then combining them into a single file to allow for processing all the records together. The files may contain the same records so need to be combined to ensure only the unique records will be processed. I have created a process job using the following method (see image Process image.png): Create a blank generic txt file to hold all the data received with the correct heading record Check to see if file 1 exists - if it does, union with the generic txt file and write back to the generic text file Check to see if file 2 exists - if it does, union with the generic txt file and write back to the generic text file Use this generic txt file in the main data job Archive all files Acknowledging that in this process there is only two files that may be received but there are others that have larger numbers of files, how then do you manage in DMS where there are more files rather than having to check for each file to add to the generic txt file? I think there must be a better way but I just don't know what it could be. Also I have found an issue with the reading in of the generic txt file in the "Data Job 1" node where the output is written correctly in the "Generate data for OSHC Dir..." as say 458 records but is not read into the "Data Job 1" in its entirety. It is stopping at the same location in the generic txt file when reading this file into the input file node within this data job. I expect that this issue is due to reading and writting to the same file but am not sure how to avoid this scenario to combine these files. Any suggestions would be greatly appreciated. Thanks, Allan.
... View more