topic Re: Reading raw data from external files or from datalines written in the program in SAS Programming

Reading raw data from external files or from datalines written in the program

MieMie — Sun, 02 Aug 2015 08:14:09 GMT

Hi everyone,

Does it matter when SAS reads raw data from external file or from datalines written in program? I thought it makes no difference at first, but I am not sure after I tried the example below.

For example, an external files contains variable-length records:

----+----1----+----2

333

4444

66666

The DATA step uses a numeric informat 5.

data numbers;

infile '/folders/myfolders/Crackman/external files.txt';

input TestNumber 5.;

run;

proc print data=numbers;

title 'Test DATA Step on External Files';

run;

When SAS read raw data from external file, the result is:

Obs TestNumber

1 333

2 55

3 55555

However, if SAS reads raw data from datalines written in the program, it has different results.

data numbers;

infile datalines;

input TestNumber 5.;

datalines;

333

4444

66666

;

proc print data=numbers;

title 'Test DATA Step';

run;

The result is

Obs TestNumber

1 22

2 333

3 4444

4 55

5 66666

So, what makes the difference in the results?

Thanks in advance.

Re: Reading raw data from external files or from datalines written in the program

Tom — Sun, 02 Aug 2015 14:01:59 GMT

When reading from inline data (CARDS or DATALINES) SAS will pad the records to a multiple of 80. I assume that is a side effect of being developed in the 70's using IBM mainframes. They have probably kept it that way to prevent breaking older programs. Try this program.

data x ;

infile cards length=l ;

input ;

put l=3. +1 _infile_ ;

cards;

1234567890

123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890

;;;;

You can use the TRUNCOVER option on the INFILE statement to make SAS not care if you try to read past the end of the line. Or use the : modifier on the INPUT statement to prevent it from reading more characters than are present.

Re: Reading raw data from external files or from datalines written in the program

KachiM — Sun, 02 Aug 2015 15:04:38 GMT

The number in SAS has always 8 bytes. The use of 5 is for writing out the output data set. So reading into SAS from

TXT file needs no Informat for a number. Try without 5 in your Input Statement. Tom has explained the reading through

datalines statement.

Re: Reading raw data from external files or from datalines written in the program

Tom — Sun, 02 Aug 2015 23:39:04 GMT

SAS stores numbers as 8 byte floating point numbers. But the length of the informat used when reading from text has nothing to do with the amount of storage that a number takes. The problem the original programmer had was when trying to read more characters (5.) than were available in the input stream. Removing the informat and letting SAS just use the default conversion from digits to numbers would work. Also adding TRUNCOVER to the INFILE statement or putting a colon in front of the INFORMAT (: 5.) would also prevent SAS from trying to read past the end of the line.

Re: Reading raw data from external files or from datalines written in the program

MieMie — Tue, 04 Aug 2015 07:18:26 GMT

Hi Tom,

Thanks very much for your explanation. However, it didn't answer my question. I guess my question was not clear. Sorry for confusion. Let me re-phrase the question.

The same records was stored in different place, an external text file or datalilnes in program. I used the same code to read and print data, and I was expecting the same results since they read from the same records. However, the results were quite different. I was wondering what causes the difference?

Thanks again.

Re: Reading raw data from external files or from datalines written in the program

Tom — Tue, 04 Aug 2015 12:26:16 GMT

The first line in my first response explains why they are different. Because SAS treats lines read from an external file differently than it treats lines processed as CARDS you are NOT reading the same data. In the external file there are no trailing blanks after the numbers and when read from CARDS there are.