topic Re: Potential bug with assigning numeric date values to variables in SAS Software for Learning Community

Potential bug with assigning numeric date values to variables

libokj — Thu, 14 Apr 2022 11:53:38 GMT

Initially, I tried to run the code below, where edate and ddate were originally non-negative integers (0, 1, 2, ...). However, when edate or ddate were originally 0, the new rewritten edate or ddate became 14736, instead of the correct 14737 from day0 = mdy(05, 07, 00). It was fine when edate or ddate were positive integers.

data csp.task3;
	set csp.task2;
	day0 = mdy(05, 07, 00);
	edate = edate + day0;
	ddate = ddate0 + day0;
run;

Then, I tried to create placeholder variables to see if the problem can be avoided.

data csp.task3;
	set csp.task2;
	day0 = mdy(05, 07, 00);
	edate0 = edate + day0;
	ddate0 = ddate + day0;
	edate = edate0;
	ddate = ddate0;
	format edate ddate day0 date9.;
run;

And I also tried this:

data csp.task3;
	set csp.task2;
	day0 = mdy(05, 07, 00);
	edate0 = intnx('day', day0, edate);
	ddate0 = intnx('day', day0, ddate);
	edate = edate0;
	ddate = ddate0;
	format edate ddate day0 date9.;
run;

Well, the problem persists in an even more bizarre way:

	edate	ddate	edate0	ddate0
1	14736	14736	14737	14737
2	14736	14738	14737	14738

I simply assigned edate0 to edate and ddate0 to ddate, and still managed to get the wrong values. I did some experiments with simple numeric values and couldn't reproduce the same error. Maybe it has to do with the special date class and some memory address problem?

Re: Potential bug with assigning numeric date values to variables

PaigeMiller — Thu, 14 Apr 2022 12:01:45 GMT

Not seeing it:

data task2;
    edate=0;
    ddate=0;
run;
data task3;
    set task2;
	day0 = mdy(05, 07, 00);
	edate = edate + day0;
	ddate = ddate + day0;
run;

Please provide (a portion of) your actual data set csp.task2 for us to use by following these instructions. DO NOT PROVIDE THE DATA IN ANY OTHER WAY.

Re: Potential bug with assigning numeric date values to variables

Astounding — Thu, 14 Apr 2022 12:12:14 GMT

You are complicating the places where something can go wrong by using a two-digit year. See if the problem remains after specifying a four-digit year in the MDY function.

Re: Potential bug with assigning numeric date values to variables

ballardw — Thu, 14 Apr 2022 14:25:16 GMT

If Edate or Ddate are supposed to be date values then why would be adding another date to them???

What do you expect to get for 01Jan1960 + 07May2000? That is what this does when edate is 0

edate + day0;
or similar for intnx
('day',day0,edate)

A value of 0 when treated as a date is 01JAN1960 which is the base for SAS dates. So when you add anything to that it is like 0+ x, which yields x. So you get the date from X.

You don't describe what you are actually trying to do so can't make suggestions.

Hint: ASSIGN date formats like DATE9. or YYMMDD10. to any variable that is supposed to be a date.

Read this:

https://communities.sas.com/t5/SAS-Communities-Library/Working-with-Dates-and-Times-in-SAS-Tutorial/ta-p/424354 has a PDF with much information about dates.

Re: Potential bug with assigning numeric date values to variables

Tom — Thu, 14 Apr 2022 14:41:00 GMT

What date did you mean to create by using ZERO as the YEAR value in the call to MDY() function? SAS dates only work with dates after January 1, 1582 It seems that SAS just assumes you meant the year 2000 instead of the value you actually passed.

What values do EDATE and DDATE0 have in the input dataset?

Your code is assuming it has a relative day value and not an actual date value. So something like 21 which would mean 21 days since some epoch date. Otherwise adding EDATE to DAY0 makes no sense.

Try checking out the range of values in the data.

proc means n min max range  data = csp.task2;
  var edate ddate0 ;
  format edate ddate0 ;
run;

Re: Potential bug with assigning numeric date values to variables

FreelanceReinh — Thu, 14 Apr 2022 16:31:56 GMT

Hello @libokj,

SAS date values (such as 14737='07MAY2000'd) are indistinguishable from any other numeric values (14,737.00 dollars, miles, etc.).
Diagnostic PUT statements often help to explain the occurrence of unexpected values.
When it comes to "bizarre" phenomena, don't trust anything but SAS formats revealing the binary content of a variable, e.g., the HEX16. format for numeric values (and the $HEXw. format for character values).

Example:

data have;
edate=-1e-11;
ddate=0;
run;

data want;
set have;
day0 = mdy(05, 07, 00);
put 'Before:' /
    (edate ddate day0) (=best22.) /
    (edate ddate day0) (=hex16.) /;
edate = edate + day0;
ddate = ddate + day0;
put 'After:' /
    (edate ddate) (=best22.) /
    (edate ddate) (=hex16.);
run;

The PUT statements write to the log:

Before:
edate=-1E-11 ddate=0 day0=14737
edate=BDA5FD7FE1796495 ddate=0000000000000000 day0=40CCC88000000000

After:
edate=14736.9999999999 ddate=14737
edate=40CCC87FFFFFFFFB ddate=40CCC88000000000

Re: Potential bug with assigning numeric date values to variables

PaigeMiller — Thu, 14 Apr 2022 16:53:30 GMT

@FreelanceReinh wrote:

When it comes to "bizarre" phenomena, don't trust anything but SAS formats revealing the binary content of a variable, e.g., the HEX16. format for numeric values (and the $HEXw. format for character values).

Great advice! I am going to bookmark this post!

Re: Potential bug with assigning numeric date values to variables

libokj — Thu, 14 Apr 2022 17:24:22 GMT

Thank you all for such quick replies. It's my first time posting here, and I have already found the SAS Community so friendly and helpful.

I did leave out some important details:

edate and ddate are numeric (integers). They represent days elapsed from day zero in a clinical trial, so simply 0, 1, 2, etc.
I was trying to transform these integer "dates" into actual date objects by adding the date for the day zero (7 May 2000), which I believe in SAS is equivalent to a numeric.

By reading @FreelanceReinh's post, I realized that this could be a problem with rounding. I'm new to SAS and found it strange that SAS "does not" differentiate between int and float types and instead only has "numeric". Thanks to your post, I've learned that there are ways to debug SAS like in other languages.

Re: Potential bug with assigning numeric date values to variables

Tom — Thu, 14 Apr 2022 18:20:17 GMT

SAS datasets have two data types. Fixed length character strings and floating point numbers.

The DATE formats will ignore fractions of a day.

822   data _null_;
823     now=today();
824     do offset=0,1,-1,-0.5,+0.5 ;
825       new=now+offset;
826       put offset= new= new= date9.;
827     end;
828   run;

offset=0 new=22749 new=14APR2022
offset=1 new=22750 new=15APR2022
offset=-1 new=22748 new=13APR2022
offset=-0.5 new=22748.5 new=13APR2022
offset=0.5 new=22749.5 new=14APR2022

Re: Potential bug with assigning numeric date values to variables

libokj — Thu, 14 Apr 2022 18:25:27 GMT

 Before:
 edate=0 ddate=0 day0=14737
 edate=0000000000000000 ddate=0000000000000000 day0=40CCC88000000000
 After:
 edate=14737 ddate=14737
 edate=40CCC88000000000 ddate=40CCC88000000000

Oops... I think that didn't turn out the way you or I expected. Their HEX and DEC codes are identical, but in the saved dataset edate magically becomes 14736 still. Could it really be a bug?

Re: Potential bug with assigning numeric date values to variables

libokj — Thu, 14 Apr 2022 18:40:21 GMT

Thank you for the information. I was aware of this feature. I did try to output the variables in HEX as @FreelanceReinhard suggested, and they were correct in the PUT output, but in the saved dataset the dates still have the same problem.

Re: Potential bug with assigning numeric date values to variables

Tom — Thu, 14 Apr 2022 18:46:50 GMT

EDATE in that print out is 14737. You can tell because it has the exact same HEX representation as DAY0.

How are you checking the new value.

How is the variable EDATE defined in the saved SAS dataset.

Did you ask SAS to eliminate some of the least significant bytes of the floating point value by asking it to only store some of the 8 bytes that are required for 64-bit floating point numbers?

840   data test;
841     length date3 3 date8 8;
842     date3=today();
843     date8=today();
844     format date: date9.;
845     put date3= date8= ;
846   run;

date3=14APR2022 date8=14APR2022
NOTE: The data set WORK.TEST has 1 observations and 2 variables.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds


847
848   data _null_;
849     set test;
850     put date3= date8= ;
851   run;

date3=13APR2022 date8=14APR2022
NOTE: There were 1 observations read from the data set WORK.TEST.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds

Re: Potential bug with assigning numeric date values to variables

libokj — Thu, 14 Apr 2022 18:52:39 GMT

I checked the saved dataset by both SAS data viewer and proc print. In decimal edate becomes 14736, and in HEX 40CCC80000000000. So in the process of saving the transformed variables to the dataset, their values did change.

Re: Potential bug with assigning numeric date values to variables

FreelanceReinh — Thu, 14 Apr 2022 18:58:08 GMT

@libokj wrote:
I checked the saved dataset by both SAS data viewer and proc print. In decimal edate becomes 14736, and in HEX 40CCC80000000000. So in the process of saving the transformed variables to the dataset, their values did change.

Yes, this will be the effect if EDATE has the (insufficient) length 3. Four bytes are the minimum for SAS date values.

Re: Potential bug with assigning numeric date values to variables

libokj — Thu, 14 Apr 2022 18:59:24 GMT

Thank you! Precision was indeed the problem. I'm new to SAS and quite unfamiliar with its data structures and features. Specifying length in the data step for edate and ddate before the rewrite solved the problem.

Re: Potential bug with assigning numeric date values to variables

libokj — Thu, 14 Apr 2022 19:09:59 GMT

Quick question tho: the official documentation says the default length in bytes for "numeric" is 8. If that's the case, there's no way that this could have happened.

Re: Potential bug with assigning numeric date values to variables

Tom — Thu, 14 Apr 2022 19:14:26 GMT

@libokj wrote:
Quick question tho: the official documentation says the default length in bytes for "numeric" is 8. If that's the case, there's no way that this could have happened.

The default length for a variable read from an existing dataset is the length that it used in that dataset.

Re: Potential bug with assigning numeric date values to variables

Tom — Thu, 14 Apr 2022 19:16:18 GMT

@libokj wrote:
Thank you! Precision was indeed the problem. I'm new to SAS and quite unfamiliar with its data structures and features. Specifying length in the data step for edate and ddate before the rewrite solved the problem.

A lot of uses are confused by LENGTH. The conflate the LENGTH (number of bytes stored in the SAS datasets) with the WIDTH of the format used to DISPLAY the value as text.

Re: Potential bug with assigning numeric date values to variables

libokj — Thu, 14 Apr 2022 19:21:33 GMT

Thanks. That's good to know... That's a cheap way to save system resources tho. I think in most languages data read from files are initialized with the default precision.

Re: Potential bug with assigning numeric date values to variables

Tom — Thu, 14 Apr 2022 19:39:35 GMT

@libokj wrote:
Thanks. That's good to know... That's a cheap way to save system resources tho. I think in most languages data read from files are initialized with the default precision.

Precision has nothing to do with it.

SAS will also default the type and storage length of a variable if you do not explicitly define it.

SAS only has two data types. Fixed length character strings and floating point numbers.

So for your EDATE variable to have been created with a storage length of 3 (or 2 if running on IBM hardware) then someone had to explicitly tell SAS to throw away the rest of the value when writing it to the disk by issuing a LENGTH statement.

A length of 3 might make sense for EDATE when it was a relative day variable. But you change the size of the values you were storing by a couple of orders of magnitude by converting it to the number of days since 1960 instead.