Help using Base SAS procedures

Reading Text File -- Not Getting All Records In SAS Data Set?

Reply
Occasional Contributor
Posts: 16

Reading Text File -- Not Getting All Records In SAS Data Set?

I'm using the following SAS code to read data in from the following text file. When I run the code the SAS data set "detail" only contains 3 records? Yet when I read the log it says it read all 7 records from the test file? Where did the other 4 records go? Help is appreciated.

er3detail.tx File contents
Last Error Message: 23-2: Invalid option name SEX
Last Warning Message: The data set WORK.WORDGENDER may be
Last Dataset Created: WORK.WORDGENDER
Test Line 4
Test Line 5
Test Line 6



SAS Code:
data detail;
length t $ 300;
infile "C:\WUSS14\er3detail.txt";
input t $ 1-300 ;
run;

SAS LOG
91
92 data detail;
93 length t $ 300;
94 infile "C:\WUSS14\er3detail.txt";
95 input t $ 1-300 ;
96 run;

NOTE: The infile "C:\WUSS14\er3detail.txt" is:
Filename=C:\WUSS14\er3detail.txt,
RECFM=V,LRECL=256,File Size (bytes)=295,
Last Modified=12Nov2010:08:05:44,
Create Time=09Nov2010:19:22:20

NOTE: 7 records were read from the infile "C:\WUSS14\er3detail.txt".
The minimum record length was 10.
The maximum record length was 140.
NOTE: SAS went to a new line when INPUT statement reached past the end of a line.
NOTE: The data set WORK.DETAIL has 3 observations and 1 variables.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
cpu time 0.00 seconds
SAS Super FREQ
Posts: 8,866

Re: Reading Text File -- Not Getting All Records In SAS Data Set?

Hi:
When I find that I have variable length records (as indicated by this note in the log):
[pre]
The minimum record length was 10.
The maximum record length was 140.
[/pre]

I turn to the $VARYING. informat to read my variable length text lines. The program below worked for me when I saved the file into c:\temp\er3detail.txt (I removed a blank line from the bottom of the file, so I only have 6 lines.

cynthia
[pre]
1662 data detail;
1663 length t $ 300;
1664 infile "C:\temp\er3detail.txt" length=lg;
1665 input t $varying. lg;
1666 run;

NOTE: The infile "C:\temp\er3detail.txt" is:
Filename=C:\temp\er3detail.txt,
RECFM=V,LRECL=256,File Size (bytes)=189,
Last Modified=12Nov2010:08:39:20,
Create Time=12Nov2010:08:39:18

NOTE: 6 records were read from the infile "C:\temp\er3detail.txt".
The minimum record length was 11.
The maximum record length was 58.
NOTE: The data set WORK.DETAIL has 6 observations and 1 variables.
NOTE: DATA statement used (Total process time):
real time 1.84 seconds
cpu time 0.00 seconds
[/pre]
Occasional Contributor
Posts: 16

Re: Reading Text File -- Not Getting All Records In SAS Data Set?

Posted in reply to Cynthia_sas
Awesome! You made my day! Thanks
Respected Advisor
Posts: 4,173

Re: Reading Text File -- Not Getting All Records In SAS Data Set?

Posted in reply to Cynthia_sas
Wouldn't a simple "truncover" in the infile statement do the same trick without having to change informats?
SAS Super FREQ
Posts: 8,866

Re: Reading Text File -- Not Getting All Records In SAS Data Set?

Patrick:
Yes, you're right. I know there are instances where TRUNCOVER would work just as well.

I happen to be a fan of $VARYING. for reading free-format text fields. I use the LG variable to get rid of "empty" lines, and with files in some formats (such as when I'm scraping HTML or scraping the LOG), I can use LG (along with other criteria) to filter out some observations from being read or being written to the output file.

This just looked like text scraping to me and so I picked $VARYING.

cynthia
Respected Advisor
Posts: 4,173

Re: Reading Text File -- Not Getting All Records In SAS Data Set?

Posted in reply to Cynthia_sas
Hi Cynthia

Thanks for your explanation.

Just to insist a bit more:
" there are instances where TRUNCOVER would work just as well"

In which instances wouldn't it work? I can't think of any?

Thanks
Patrick
SAS Super FREQ
Posts: 8,866

Re: Reading Text File -- Not Getting All Records In SAS Data Set?

Patrick:
I am a "never say never" and "never say always" kind of girl. When I am text scraping I use $VARYING. You may use TRUNCOVER. This is "probably" an instance where either would work for the data that was posted. However, I remain a firm fan of $VARYING.

As for your question...here's an instance:
http://support.sas.com/kb/5/411.html

And I suspect I stopped using TRUNCOVER because of this or similar behavior in earlier versions of SAS:
http://support.sas.com/kb/32/889.html

But as I said, I almost always use $VARYING. for text scraping. TRUNCOVER would not have been my suggestion unless there were more variables being read in the INPUT statement than just a possible 300 character line of text.

Just a preference. Do whatever you prefer. There's usually (note the "usually") more than one way to accomplish the same task with SAS.

cynthia
Respected Advisor
Posts: 4,173

Re: Reading Text File -- Not Getting All Records In SAS Data Set?

Posted in reply to Cynthia_sas
Cynthia
Learnt something new! Thanks!
I just knew that it's worth bothering you a bit more :-)
Patrick
Ask a Question
Discussion stats
  • 7 replies
  • 875 views
  • 0 likes
  • 3 in conversation