Building models with SAS Enterprise Miner, SAS Factory Miner, SAS Visual Data Mining and Machine Learning or just with programming

SAS read text file(s) with unknown delimiters, and unknown lengths?

Reply
Frequent Contributor
Posts: 90

SAS read text file(s) with unknown delimiters, and unknown lengths?

I am searching my file system mining for metadata, am looking at all old txt files in this iteration.  Thus I am in this case trying to read txt files with per file an unknown delimiter, and unknown length.  Is there a preferred method (function) to go about this?  I was thinking a whole line at a time, but am open to ideas. -Keith

(I have already done the work for many other file extensions with easily extractable metadata)

Super User
Posts: 17,851

Re: SAS read text file(s) with unknown delimiters, and unknown lengths?

Proc Import and then use the code created. Let the computer take the first guess.
Respected Advisor
Posts: 4,651

Re: SAS read text file(s) with unknown delimiters, and unknown lengths?

I would define a small set of possible delimiters and do a character count for each candidate delimiter within the first, say, 20 lines of the file. If none of the delimiters appears, I would assume that the file is delimited by spaces.

 

Another more subtle approach would use the fact that delimiters should occur at the same frequency on every line.

PG
Ask a Question
Discussion stats
  • 2 replies
  • 279 views
  • 0 likes
  • 3 in conversation