PIPE imports only first line in each file

[shl007]

Hi - I am using a PIPE to import multiple text files. The PIPE does recognize all files in the directory but is only importing the first line in each file? For example, if files A & B are recognized by the pipe, only the first line in A and the first line in B are imported. The file is delimited by '^'. Any ideas why only the first line of each file is imported? Below is the code I am using, including a DO-WHILE loop. I am using this as part of a macro.

filename output pipe %sysfunc(quote(ls -a "&pipepath"*_*.txt));

DATA WORK.testupd;
    INFILE output TRUNCOVER ;
    INPUT name $250.;/*Pull pipe in*/
    fil2read=name;
	INFILE dummy FILEVAR=fil2read END=last FIRSTOBS=1  LENGTH=len truncover
		 /*LRECL=292*/
        ENCODING="LATIN1"
        TERMSTR=CRLF
        DLM='^' 
        MISSOVER
        DSD ;

    LENGTH
        F1               $ 28	/* field1*/
        F2               $ 1	/* field2 */
        F3               $ 8  /* field3 */
        F4               $ 10  /* field4 */
        F5               $ 60	/* field5 */
        F6               $ 50	/* field6 */
;
    FORMAT
        F1               $CHAR28.
        F2               $CHAR1.
        F3               $CHAR8.
        F4               $CHAR10.
        F5               $CHAR60.
        F6               $CHAR50.
;
    INFORMAT
        F1               $CHAR28.
        F2               $CHAR1.
        F3               $CHAR8.
        F4               $CHAR10.
        F5               $CHAR60.
        F6               $CHAR50.
 ;
	do while (not last);
	INPUT
        F1               : $CHAR28.
        F2               : $CHAR1.
        F3               : $CHAR8.
        F4               : $CHAR10.
        F5               : $CHAR60.
        F6               : $CHAR50.
;
		Rundt  = today();
		format Rundt date10.;

		filenm = fil2read;

		output;
	end;
RUN;

DATA WORK.testupd;
    INFILE output TRUNCOVER ;
    INPUT name $250.;/*Pull pipe in*/
    fil2read=name;
	INFILE dummy FILEVAR=fil2read END=last FIRSTOBS=1  LENGTH=len truncover
		 /*LRECL=292*/
        ENCODING="LATIN1"
        TERMSTR=CRLF
        DLM='^' 
        MISSOVER
        DSD ;

    LENGTH
        F1               $ 28	/* field1 */
        F2               $ 1	/* field2*/
        F3               $ 8  /* field3*/
        F4               $ 10  /* field4 */
        F5               $ 60	/* field5 */
;
    FORMAT
        F1               $CHAR28.
        F2               $CHAR1.
        F3               $CHAR8.
        F4               $CHAR10.
        F5               $CHAR60.
;
    INFORMAT
        F1               $CHAR28.
        F2               $CHAR1.
        F3               $CHAR8.
        F4               $CHAR10.
        F5               $CHAR60.
;
	do while (not last);
	INPUT
        F1               : $CHAR28.
        F2               : $CHAR1.
        F3               : $CHAR8.
        F4               : $CHAR10.
        F5               : $CHAR60.
;
		Rundt  = today();
		format Rundt date10.;

		filenm = fil2read;

		output;
	end;
RUN;

[shl007]

I inadvertently copied my code above twice; just refer to one of the snippets.

[Kurt_Bremser]

You are working on UNIX. Are you sure the files use the Windows line separator CRLF, and not the UNIX-typical LF?

[shl007]

Hi Kurt - not sure to your question. How can I tell which LF character is being used? If it is using a different-than-expected LF character, is there a way I can account for that in the code? Thanks for any tips!!

[ballardw · Accepted solution]

---

@shl007 wrote:

Hi Kurt - not sure to your question. How can I tell which LF character is being used? If it is using a different-than-expected LF character, is there a way I can account for that in the code? Thanks for any tips!!

---

Your code shows the option TERMSTR=CRLF. Which tells SAS to read the file with the Windows standard CarriageReturn/LineFeed character combination as the end of line marker. This is an operating system default. If the file is actually a UNIX file which would use LF as the end of line marker then you have told the program with the CRLF that there is only one line. If the LF should be the end of line character than either use Termstr=LF or remove the Termstr option entirely which will then use the default for the operating system involved.

[SASKiwi]

@shl007  - What happens if you remove TERMSTR=CRLF?