pipe delimietd file with no specific layout

[Tal]

this  really  works like  a  charm Tom,wow. Thanks a  lot!

One last thing i'd ask  is  why do you think for a  few  out of 1000 records  the ;last  field  gets  cut off  like  i ve shown in the  pic?

i checked  those  2 particular reocrds  in the  text  file and  they  have the  full dates there.

[Tom]

Check the notes in the log for data steps that read the file. It is possible the lines were truncated.

Also it might be that the code to append the lines added two  extra spaces. Make that last column wider to see if the problem is that the data looks like:

2014-01-  05

Or perhaps worse that it looks like there is another whole line stuck on the end of that line.

[Tal]

i think  this code  :

data _null_

infile '/path/temp.txt"  end=eof;

 input ;

if _n_>1 then do;

if countw(_infile_,'|','mq') > 3 then put / @;

else put ' ' @;

end;

put _infile_ @;

if eof then put;

run;

truncuates the "notes" field at some point but  dont  undrstand  why.

I have attached  a few  records and  the  1st  record  is  trunctated at the "150%" and  along  with it  the 21th field is assigned a  missing value

whereas the  second record  is appended  properly.

Also the  3rd record  which is longer than the 1st is read in properly

[Tom]

Seems to me that you are still trying to edit this TEXT file using EXCEL.  This time you posted a file with a lot of TAB characters on the ends of the lines instead of COMMAs.  Edit the file using a TEXT editor. The SAS program editor will do if you don't have anything else!

To really see what is happening post the LOG which will include very useful information.

102  filename dat temp ;
103  data _null_;
104    infile "&path/temp (2).txt" end=eof lrecl=32767;
105    file dat lrecl=32767;
106    input ;
107    if _n_>1 then do;
108      if countw(_infile_,'|','mq') > 2 then put / @;
109      else put '  ' @;
110    end;
111    put _infile_ @;
112    if eof then put;
113  run;

NOTE: The infile ...
      RECFM=V,LRECL=32767,File Size (bytes)=889,
      Last Modified=04May2017:18:03:35,
      Create Time=04May2017:18:03:34

NOTE: The file DAT is:

      Filename=...
      RECFM=V,LRECL=32767,File Size (bytes)=0,
      Last Modified=04May2017:18:06:00,
      Create Time=04May2017:18:06:00

NOTE: 6 records were read from the infile ".../temp (2).txt".
      The minimum record length was 0.
      The maximum record length was 289.
NOTE: 3 records were written to the file DAT.
      The minimum record length was 278.
      The maximum record length was 303.

Depending on the version of SAS you are using you might be defaulting to line lengths of only 256.  All of the extra tabs could also take up space, especially if they were expanding into 8 spaces each.  If I add the EXPANDTABS option to the INFILE statement I get these notes on the line lengths.

NOTE: 6 records were read from the infile ".../temp (2).txt".
      The minimum record length was 0.
      The maximum record length was 289.
NOTE: 3 records were written to the file DAT.
      The minimum record length was 344.
      The maximum record length was 434.

[Tal]

the  last file i sent  was a  copy and  paste  from my original csv  to a  text editor  and maybe  that s why it got  changed again

but   you were right  i was missing  the  lrecl=32767

Works  fine now

Million thanks Tom

Enjoy the weekend

[Tal]

i just wonder  what does  the following  do:

if countw(_infile_,'|','mq') > 3 then put / @;

else put ' ' @;

[Tom]

---

@Tal wrote:

i just wonder  what does  the following  do:

 if countw(_infile_,'|','mq') > 3 then put@ / @;

else put ' '@ @;

---

If more than three fields on the current input line then output a linebreak else output a space and in both cases keep the current (or new current) line open for more characters.

The _INFILE_ is the input buffer. It is the current line read by the latest INPUT statement.

The COUNTW() function counts words. This call is using '|' as the delimiter between words and the M modifier says to treat adjacent delimiters to mean there is an empty word. The Q modifier says to ignore delimiters inside of quotes.  

The '/' modifier on a PUT statement means go to the next line. So it writes an end of line ( CR+LF) to the output file.

The '@' modifier on a PUT statement means to keep the cursor at the current location. It prevents this PUT statement from writing the end of line.

[Tal]

thanks