Concatenating Several Raw Data Files into One

[ismahero2]

Hi,

I have several txt files of raw data that I need to concatenate into one single file.  The problem is that the data in some of these files is larger than 32K.  So when I tried to do the traditional infile option to place the data in a file, is truncating part of the data.  Any idea how could I use SAS to take those files and put them together into one single raw file without any truncation?

[Kurt_Bremser]

The maximum LRECL for Windows and UNIX is 1G (1 Gigabyte), so lines longer than 32K should not be a problem.

The only thing of concern that is limited to 32767 is the length of a single character variable.

[ballardw]

---

@ismahero2 wrote:

Hi,

I have several txt files of raw data that I need to concatenate into one single file.  The problem is that the data in some of these files is larger than 32K.  So when I tried to do the traditional infile option to place the data in a file, is truncating part of the data.  Any idea how could I use SAS to take those files and put them together into one single raw file without any truncation?

---

Details may be nice.

If you want to concatenate TEXT files there should be lots of system tools that do that without any SAS involvement.

If you want to concatenate SAS data sets that is another question. And if you are having variables that need to hold more than 32K characters that is a significantly different issue. If the issue is creating SAS data sets then your explanation needs some expansion. Such as, do you have any documentation describing what these text files contain in terms of  row descriptions like column order, types and lengths? That would likely be the place to start.

[RichardDeVen]

Are these several text files from which you will be inputting line-wise ?  If so, what variables are you inputting ?

Or are they binary files that somehow when concatenated make sense ?  If binary, how are you reading data from the result ?

Several text files can be read line by line using a INFILE statement that specifies a wildcarded filename.  Use LRECL if a single line can be very wide.

infile 'c:\data\txt\2020-*.txt' lrecl=1000000;
input ...;

[Reeza]

Are you modifying the files in any manner?

Do you have XCMD enabled?

If so, I would consider using system commands such as CAT instead.

---

@ismahero2 wrote:

Hi,

I have several txt files of raw data that I need to concatenate into one single file.  The problem is that the data in some of these files is larger than 32K.  So when I tried to do the traditional infile option to place the data in a file, is truncating part of the data.  Any idea how could I use SAS to take those files and put them together into one single raw file without any truncation?

---