topic Re: E8601dt informat does not like missing values in SAS Programming

E8601dt informat does not like missing values

waveson — Thu, 21 Jan 2021 08:42:51 GMT

This seems to me quite peculiar. When reading datetimes with E8601DT. informat like this:

data test; length date1 8 date2 8 date3 8; informat date1 e8601dt. date2 e8601dt. date3 e8601dt.; infile datalines DSD MISSOVER; input date1 date2 date3; cards; 2020-09-27 00:00:01.000,,2020-09-27 00:00:00.000 ; run;

...I'm getting a note:

NOTE: Invalid data for date2 in line 34 25-25.
With the whole dataset I'm eventually getting a WARNING that there were too many of these cases throughout the file. I don't want to supress these warnings, because I do not trust that the source too much. I just would like SAS to acknowledge that the missing value is fine and carry on.

Just like it does with other informats, like datetime22. which does not seem to have any issues with missing values:

data test;
length date1 8 date2 8 date3 8;
informat date1 datetime22. date2 datetime22. date3 datetime22.;
infile datalines DSD MISSOVER;
input date1 date2 date3;
cards;
09sep2020 00:00:01.000,,09sep2020 00:00:00.000
;
run;

Do you have any suggestions?
(SAS version: 9.4_M6)

Re: E8601dt informat does not like missing values

ChrisNZ — Thu, 21 Jan 2021 00:33:10 GMT

These formats are recent, and you found a defect.

1. Report to SAS Tech Support so they can open a defect entry.

2. I tried 100k values and still don't get a warning. How many missing values do you have?

3. To work around the error, something like this should work:

data T;
  informat DATE1 DATE2 DATE3 e8601dt.;
  array DT[*] DATE1-DATE3;
  infile cards dsd missover;
  do I=1 to dim(DT);
    input DT[I] 1. @;   
    if DT[I]=2 then do; 
      putlog DT[I]=; 
      input +(-1) DT[I] @;  
      putlog DT[I]=; 
    end;
  end;
cards;
2020-09-27 00:00:01.000,,2020-09-27 00:00:00.000
run;

But I am getting unexpected results (9.4M2 under Windows). The second input always generates a note even if the test is false (putlog is not executed).

If I replace the test with if 0 then no note is generated.

30         data T;
31           informat DATE1 DATE2 DATE3 e8601dt.;
32           array DT[*] DATE1-DATE3;
33           infile cards dsd missover;
34           do I=1 to dim(DT);
35             input DT[I] 1. @;
36             if DT[I]=2 then do;
37               putlog DT[I]=;
38               input +(-1) DT[I] @;
39               putlog DT[I]=;
40             end;
41           end;
42         cards;

DATE1=2
DATE1=1916784001
NOTE: Invalid data for DATE2 in line 43 25-25.
DATE3=2
DATE3=1916784000
RULE:      ----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0                     
43         2020-09-27 00:00:01.000,,2020-09-27 00:00:00.000
DATE1=1916784001 DATE2=. DATE3=1916784000 I=4 _ERROR_=1 _N_=1
NOTE: The data set WORK.T has 1 observations and 4 variables.
NOTE: Compressing data set WORK.T increased size by 100.00 percent. 
      Compressed is 2 pages; un-compressed would require 1 pages.
NOTE: DATA statement used (Total process time):
      real time           0.07 seconds
      user cpu time       0.00 seconds
44         run;

45         
46         data T;
47           informat DATE1 DATE2 DATE3 e8601dt.;
48           array DT[*] DATE1-DATE3;
49           infile cards dsd missover;
50           do I=1 to dim(DT);
51             input DT[I] 1. @;
52             if 1=2 then do;
53               putlog DT[I]=;
54               input +(-1) DT[I] @;
55               putlog DT[I]=;
56             end;
57           end;
58         cards;

NOTE: The data set WORK.T has 1 observations and 4 variables.
NOTE: Compressing data set WORK.T increased size by 100.00 percent. 
      Compressed is 2 pages; un-compressed would require 1 pages.
NOTE: DATA statement used (Total process time):
      real time           0.07 seconds

That's weird.

Re: E8601dt informat does not like missing values

andreas_lds — Thu, 21 Jan 2021 06:03:11 GMT

Unfortunately the problem can be reproduced using 9.4m7:

NOTE: Invalid data for date2 in line 34 25-25.
REGEL:     ----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0                     
34         2020-09-27 00:00:01.000,,2020-09-27 00:00:00.000
date1=1916784001 date2=. date3=1916784000 _ERROR_=1 _N_=1
NOTE: The data set WORK.TEST has 1 observations and 3 variables.
NOTE:  Verwendet wurde: DATA statement - (Gesamtverarbeitungszeit):
      real time           0.01 seconds
      cpu time            0.01 seconds
      

35         ;
36         
37         %put &=SYSVLONG;
SYSVLONG=9.04.01M7P080520

Re: E8601dt informat does not like missing values

Kurt_Bremser — Thu, 21 Jan 2021 09:01:17 GMT

I also consider this a bug, so open a track with Technical Support.

Here is a workaround that can help for the moment:

proc format;
invalue my8601dt
  " " = .
  other = [e8601dt.]
;
run;

data test;
length date1 8 date2 8 date3 8;
informat date1 date2 date3 my8601dt.;
infile datalines dsd truncover;
input date1 date2 date3;
cards;
2020-09-27 00:00:01.000,,2020-09-27 00:00:00.000
;

Re: E8601dt informat does not like missing values

FreelanceReinh — Thu, 21 Jan 2021 10:15:02 GMT

@ChrisNZ wrote:

But I am getting unexpected results (9.4M2 under Windows). The second input always generates a note even if the test is false (putlog is not executed).

@ChrisNZ: Good news: It's not the second, but the first input that (correctly) generated the note

NOTE: Invalid data for DATE2 in line 43 25-25.

when trying to read the (second) comma into a numeric variable using formatted input.

Re: E8601dt informat does not like missing values

ChrisNZ — Fri, 22 Jan 2021 04:40:42 GMT

> It's not the second, but the first input that (correctly) generated the note

Ah yes! Thank you!

Re: E8601dt informat does not like missing values

waveson — Fri, 22 Jan 2021 09:49:58 GMT

I have just opened a track at SAS Tech Support.
I have also used @Kurt_Bremser 's suggestion as a temporary solution in my project.

Thanks everyone!

Re: E8601dt informat does not like missing values

waveson — Wed, 10 Mar 2021 13:04:41 GMT

I have received a response from SAS tech support:

Fixing that issue is on the list for the next maintenance - we still cannot give any date for the release of that version.