DATA Step, Macro, Functions and more

Import multiple specific .txt files

Reply
Contributor
Posts: 69

Import multiple specific .txt files

I have SAS EG and everything is directed through our SAS Server.

 

I already found out that I cannot point my code to a shared drive within our organization that contains multiple zipped folders with multiple various .txt files and get all of what I need to import.

 

So, the way around it is to move all the zipped to my local drive. That took 5 hours. Then I unzipped everything and that took about 3 hours. Now I have 10 folders on my local hard drive and just starting with 1 of the folders. There are 288 .txt files in this particular folder. Some of them have an extension of flaggedevents.txt and the others details.txt

 

I want to import all of the .txt files that have the extension details.txt.  Is there a way I can do this with SAS code??? The reason I want to use the details.txt is because the file names vary. Like adol_well_care_16_ch_details.txt and the next is adult_bmi_assmt_details.txt

Super User
Posts: 17,840

Re: Import multiple specific .txt files

There isn't a wild card method to refer to files with a common suffix AFAIK.

 

Can you obtain a list of the file names? If you can't use pipe or x commands, you can use the methods here:

http://www2.sas.com/proceedings/forum2007/008-2007.pdf

 

Do all of the files have the same structure?

If so, you can use the methods here:

https://communities.sas.com/t5/SAS-Communities-Library/How-do-I-write-a-macro-to-import-multiple-tex...

 

Contributor
Posts: 69

Re: Import multiple specific .txt files

Yes Reeza, I see all the file names. In all from all these folders I have 1,102 .txt files to import. For the most part all the columns SHOULD be the same. I wished they would just get this into a database already. Last time I had to split it up. One person did our Medicare the other Medicaid. Different populations although there are duals. Took 3 months for them to complete the project. Now, Medicare still has the .txts but thankfully they built a datalab to hold the files and I did all this teradata coding and now the code runs in 1 hour. So from 3 months to 1 hour. Not too shabby. But Medicaid is HUGE. Our population is gigantic and they have yet to build a lab. So I have to use these .txt files and told them I sincerely hope nothing changes like column headers or datatypes because I am going to create SAS code to import all this stuff, then work it because it does have to be cleaned and if the next time we run (once a quarter is the ask) if they change one little thing the SAS automation portion that cleans and does all the manipulating will be thrown out of whack. It is the same with the lab. If they change something. Then I get oh, so it isn't truly automated??? I want to scream clueless people. The only way to automate and account for changes of incoming stuff is to create a brain that can say oh if the datatype is now this then do that. But if datatype stays the same do this. They don't get they are talking about 1,102 separate .txt files (as of now), and if it were as easy a balancing a checkbook then everyone would and could do it. thanks. I will look at the links.

Ask a Question
Discussion stats
  • 2 replies
  • 156 views
  • 0 likes
  • 2 in conversation