Can you verify that you are using SPDE (you write "SPD" only), or perhaps SPDS? Generally speaking, those engines rock when it comes to inserts, modify/update not as much. When you have that many observations in your transaction data set, sometimes a brute force MERGE or JOIN could be quicker. ... View more

Can you post some real data? It is depend on the pattern that start a new line . https://communities.sas.com/t5/SAS-Programming/Input-a-CSV-file-with-text-fields-that-contain-line-breaks/m-p/821603 https://communities.sas.com/t5/SAS-Programming/Proc-import-issue-where-the-data-is-both-comma-separated-amp/m-p/833697 https://communities.sas.com/t5/SAS-Enterprise-Guide/How-to-properly-read-a-csv-file/m-p/855247 ... View more

@AnnLyn I assume you want to rename the file to a static name so you can use it with an external file reader transformation. It's years ago that I've used DIS but if I remember right then once you've created the external file metadata object it's possible to amend it to use a filename with a column modifier (class_:) which then would work for any external file that fits the pattern. I can't remember if this tweak needs to be done in the external file metadata or the file reader transformation. If you look at the generated code and then play a bit with DIS (=change something and inspect where the generated code changes) you should be able to figure it out. ... View more

They delete metadata libraries not physically. ... View more

You could start with the SAS supplied macro %MDSECGO and then amend the script as you need it. https://go.documentation.sas.com/doc/en/bicdc/9.4/bisecag/n1imi9yor986fsn1o25wd6l582ex.htm    Your need is not that uncommon and if you search a bit you should find something that comes already close to what you need. Here for example.   ... View more

it works great. Thank you so much for your help. I am very happy with your solution. ... View more