topic Reading multiple occurrances of the same fields in SAS - using @ and @@ in SAS Programming

Reading multiple occurrances of the same fields in SAS - using @ and @@

ramkal21 — Sun, 24 Dec 2023 18:00:06 GMT

Hi All,

I have a scenario to read below Input data

Field1 Field2 Field3 Field4 Field2 Field3 Field 4 Field2 Field3 Field4 and so on till 78 times

and write it as,

Field1 Field2 Field3 Field4

Field1 FIeld2 Field3 Field4

I tried writing the INPUT statements as below

Input Field1 @;

Input Field2 Field3 Field4 @@;

However, it seems to go in a loop and job ended up failing due to space issue with SAS dataset. The input file has less than 100,000 records and sets of Field2, Field3 Field4 is 78. Output records count should be 78*100,000 = 7,800,000

So am assuming the issue with my code and not with the space issue.

Could you please clarify?

I am getting below error in the logs

ERROR: Insufficient space in file WORK.SASDAT1.DATA.
ERROR: File WORK.SASDAT1.DATA is damaged. I/O processing did not complete.
NOTE: 1 record was read from the infile INPFIL1.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: Due to ERROR(s) above, SAS set option OBS=0, enabling syntax check mode.
This prevents execution of subsequent data modification statements.
WARNING: The data set WORK.MAD900WFL may be incomplete. When this step was stopped there were 1712581607 observations and 10
variables.

Thanks your help!

Happy Holidays

Ram

Re: Reading multiple occurrances of the same fields in SAS - using @ and @@

Tom — Sun, 24 Dec 2023 18:24:50 GMT

Post the actual SAS log so we can see what code you ran that generated that error.

Is the goal to read from a TEXT file and write to another TEXT file?

data _null_;
  infile 'original.txt' flowover ;
  input var1-var78 ;
  file 'newfile.txt' ;
  put (var1-var78) (:best32. :best32. :best32. :best32. / );
run;

You do know that 78 is not a multiple of 4.

211  data _null_;
212    infile in flowover ;
213    input var1-var78 ;
214    file log ;
215    put (var1-var78) (:best32. :best32. :best32. :best32. / );
216  run;

NOTE: The infile IN is:
      (system-specific pathname),
      (system-specific file attributes)

1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
21 22 23 24
25 26 27 28
29 30 31 32
33 34 35 36
37 38 39 40
41 42 43 44
45 46 47 48
49 50 51 52
53 54 55 56
57 58 59 60
61 62 63 64
65 66 67 68
69 70 71 72
73 74 75 76
77 78
NOTE: 1 record was read from the infile (system-specific pathname).
      The minimum record length was 224.
      The maximum record length was 224.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds

Re: Reading multiple occurrances of the same fields in SAS - using @ and @@

Tom — Mon, 25 Dec 2023 02:50:30 GMT

Do you mean you have 1+78*3 = 235 values on each line of text?

So if the goal is to create a DATSET and not another TEXT file then you should be able to generate one observation per repetition using a DO loop.

Does every line in the source file have all 235 values? If not then you need some type of stopping rule.

data want;
  infile 'myfile.txt' length=ll column=cc truncover;
  input field1 @;
  do rep=1 by 1 until( cc>ll);
    input field3-filed4 @ ;
    output;
  end;
run;

Re: Reading multiple occurrances of the same fields in SAS - using @ and @@

mkeintz — Mon, 25 Dec 2023 04:54:55 GMT

I presume you mean that each line of your input file has field1, followed by 78 triplets of field2/field3/field4, yes?

Then something like (say you have only space separated numeric values)

data want;
  infile 'name of your input file' ;
  input field1 @;
  do _n_=1 to 78;
    input field2 field3 field4 @;
    output;
  end;
run;

Re: Reading multiple occurrances of the same fields in SAS - using @ and @@

Astounding — Mon, 25 Dec 2023 18:31:07 GMT

Your code keeps reading the first line of data. Nothing in your program allows the software to move to the second line. While several possible solutions have been mentioned in passing, the right solution would depend on us knowing a bit more about the structure of the incoming data.

Re: Reading multiple occurrances of the same fields in SAS - using @ and @@

ramkal21 — Tue, 26 Dec 2023 03:54:22 GMT

Thanks much @mkeintz . Yes, this is exactly what i needed. It worked!