topic Re: SAS read text file(s) with unknown delimiters, and unknown lengths? in SAS Data Science

SAS read text file(s) with unknown delimiters, and unknown lengths?

kjohnsonm — Thu, 17 Dec 2015 21:07:51 GMT

I am searching my file system mining for metadata, am looking at all old txt files in this iteration. Thus I am in this case trying to read txt files with per file an unknown delimiter, and unknown length. Is there a preferred method (function) to go about this? I was thinking a whole line at a time, but am open to ideas. -Keith

(I have already done the work for many other file extensions with easily extractable metadata)

Re: SAS read text file(s) with unknown delimiters, and unknown lengths?

Reeza — Fri, 18 Dec 2015 03:03:09 GMT

Proc Import and then use the code created. Let the computer take the first guess.

Re: SAS read text file(s) with unknown delimiters, and unknown lengths?

PGStats — Fri, 18 Dec 2015 04:18:58 GMT

I would define a small set of possible delimiters and do a character count for each candidate delimiter within the first, say, 20 lines of the file. If none of the delimiters appears, I would assume that the file is delimited by spaces.

Another more subtle approach would use the fact that delimiters should occur at the same frequency on every line.