BookmarkSubscribeRSS Feed
ChrisNZ
Tourmaline | Level 20

After many time-consuming message exchanges, I am told by tech support that reading

X=0.5000000000000000000000000   (that's L=23 in our discussion)

as something else than 0.5 is expected behaviour.

 

If someone else wants to try and report this...

Sigh...

 

 

 

 

 

Nancy05
Quartz | Level 8

I tried your code, and got this result:

V NB
0 .

 

I am running on SAS EG 7.12 HF3 (7.100.34000) (64-bit) on window 8.0.

 

 

ChrisNZ
Tourmaline | Level 20

Will no one try to push this?

If this is expected behaviour, the expectations are very low, and I hope I'll be retired before they're met... 🙂

Tom
Super User Tom
Super User

Did you open a ticket with SAS support?  What did they say?

 

ChrisNZ
Tourmaline | Level 20

I was told by tech support that reading

X=0.5000000000000000000000000   (that's L=23 in our discussion)

as not 0.5 is expected behaviour.

 

ChrisNZ
Tourmaline | Level 20

Just to prove my point (not to bring criticism, we are not in a blaming game) as I reckon that's hard to believe, here is my last email after a long series detailing the issue and getting nowhere:

1) If I run this under Windows:

data T;

  NB=O.5OOOOOOOOOOOOOOOOOOOOOOOO;    (replace with zeros)

run;

 SAS will not store the correct number.

 2) If I run the same code on a different platform, or if I add or remove zeros, the correct number is stored.

 How 1) is acceptable is beyond me. How 1) and 2) are acceptable even more so.

If you somehow think this is expected behaviour, please tell me and close the track.

 

and the nonsensical reply:

Speaking to the developer, this is expected behaviour due to the algorithms being used to store numbers that are using more bytes than can be accurately stored.

 

ChrisNZ
Tourmaline | Level 20

So no one to pick up the ball?

I'll open a suggestion entry if no one is interested. I can't let this die off like this.

mike_jones_SAS
SAS Employee

I support this routine on SAS for Windows (both 32-bit and x64 Windows).

 

This issue is not OS related. The issue pertains to floating point representation on the x64 processors. IEEE floating point representation can only represent 15 digits of accuracy. Sometimes you can see 16 digits of accuracy. In the initial example, the 15 digits of accuracy is easily exceeded even though there are cases where the result is the exact representation for 0.5.

 

The routine used to compute the result is different on Windows than on any other host (Linux, UNIX, AIX, etc.). The routine is written in x64 assembly to maximize performance since almost every alpha-numeric in SAS is processed by this routine. The algorithm used has existed since I wrote the routine back in v6.03 SAS for PC-DOS (16-bit). Only the instruction set has changed.

 

Here’s an example to show this issue:

 

33   data _null_;

34   x=0.5000000000000000000000000;

35   y=0.5000000000000000000000001;

36   put x= hex16. y= hex16.;

37   run;

 

x=3FDFFFFFFFFFFFFF y=3FDFFFFFFFFFFFFF

NOTE: DATA statement used (Total process time):

      real time           0.00 seconds

      cpu time            0.00 seconds

 

In this case, the result is slightly less than 0.5 for x as well as y. However, y should be different. A comparison of these two variables would be equal but should not as we know.

 

Here’s another example to show this issue:

 

38   data _null_;

39   x=0.500000000000000000000000;

40   y=0.500000000000000000000001;

41   put x= hex16. y= hex16.;

42   run;

 

x=3FE0000000000000 y=3FE0000000000000

NOTE: DATA statement used (Total process time):

      real time           0.00 seconds

      cpu time            0.00 seconds

 

In this case, the result is exactly 0.5 for x as well as y by reducing the number of trailing zeros. However, y should be different. A comparison of these two variables would be equal again but should not as we know.

 

Now, let’s reduce the trailing zeros to where the least significant digit can be seen in the result.

 

58   data _null_;

59   x=0.5000000000000000;

60   y=0.5000000000000001;

61   put x= hex16. y= hex16.;

62   run;

 

x=3FE0000000000000 y=3FE0000000000001

NOTE: DATA statement used (Total process time):

      real time           0.00 seconds

      cpu time            0.01 seconds

 

A comparison for x and y in this case would be not equal and rightfully so.

When comparing floating point numbers, the COMPFUZZ function is recommended.

 

  http://support.sas.com/documentation/cdl/en/lefunctionsref/67960/HTML/default/viewer.htm#p0ifledavu3...

 

ChrisNZ
Tourmaline | Level 20

Thank you for your reply @mike_jones_SAS, I much appreciate your taking the time to provide your expert input here.

 

I fail to see how the floating point representation on the x64 CPU or more generally IEEE floating point representation have anything to do with this.

SAS Linux on x64 and doesn't have the issue either apparently, according to @Tom.

POWER processors also follow the IEEE Standard for Floating-Point arithmetic and yet SAS running on this platform does not exhibit this behaviour.

 

The floating point representation for 0.5 is 3FE0000000000000.

 

And for some reason in a few specific, reproducible cases, SAS reads this number wrongly. SAS could easily store the correct value if it tried. But SAS chooses to store an incorrect value instead, because that's what it's read.

SAS reads wrongly:

1- On Windows only

2- Depending on random changes in the number of non-significant zeros.

0.50000000000000000000000000 is       read correctly by the SAS interpreter

0.5000000000000000000000000   is not read correctly by the SAS interpreter

0.500000000000000000000000     is       read correctly by the SAS interpreter

 

All three values are hex value 3FE0000000000000, easily stored in IEEE floating point representation.

All values are read correctly on all SAS platforms, except for 1)Windows 2) if there are 24 trailing zeros. 23 and 25 are fine.

Why?

 

 

 

 

 

PGStats
Opal | Level 21

Hi @mike_jones_SAS, it's great having you in this discussion.

 

There is a qualitative difference between reading the same value for numbers that differ by less than a small amount and reading different values for numbers that are the same. Further, I would expect that numbers that round to the same value would be represented by the same value.
Take Pi for example. Whether I expand it to 20 or 30 decimals, it should always be equal to CONSTANT("PI") in a 15-16 decimal representation. But it isn't:

 

 

63   data _null_;
64   decimals = 10;
65   do pi =
66        3.1415926536
67       ,3.14159265359
68       ,3.141592653590
69       ,3.1415926535898
70       ,3.14159265358979
71       ,3.141592653589793
72       ,3.1415926535897932
73       ,3.14159265358979324
74       ,3.141592653589793238
75       ,3.1415926535897932385
76       ,3.14159265358979323846
77       ,3.141592653589793238463
78       ,3.1415926535897932384626
79       ,3.14159265358979323846264
80       ,3.141592653589793238462643
81       ,3.1415926535897932384626434
82       ,3.14159265358979323846264338
83       ,3.141592653589793238462643383
84       ,3.1415926535897932384626433833
85       ,3.14159265358979323846264338328
86       ,3.141592653589793238462643383280
87       ,3.1415926535897932384626433832795
88       ,3.14159265358979323846264338327950
89       ,3.141592653589793238462643383279503
90       ,3.1415926535897932384626433832795029
91       ,3.14159265358979323846264338327950288
92       ,3.141592653589793238462643383279502884
93       ,3.1415926535897932384626433832795028842
94       ,3.14159265358979323846264338327950288420
95       ,3.141592653589793238462643383279502884197
96       ,3.1415926535897932384626433832795028841972
97       ,3.14159265358979323846264338327950288419717
98       ,3.141592653589793238462643383279502884197169
99       ,3.1415926535897932384626433832795028841971694
100      ,3.14159265358979323846264338327950288419716939
101      ,3.141592653589793238462643383279502884197169400
102      ,3.1415926535897932384626433832795028841971693994
103      ,3.14159265358979323846264338327950288419716939938
104      ,3.141592653589793238462643383279502884197169399375
105      ,3.1415926535897932384626433832795028841971693993751
106      ,3.14159265358979323846264338327950288419716939937510;
107      eq = PI = constant("PI");
108      put decimals= eq= PI= hex16.;
109      decimals + 1;
110      end;
111  run;

decimals=10 eq=0 pi=400921FB544486E0
decimals=11 eq=0 pi=400921FB54442EEA
decimals=12 eq=0 pi=400921FB54442EEA
decimals=13 eq=0 pi=400921FB54442D28
decimals=14 eq=0 pi=400921FB54442D11
decimals=15 eq=1 pi=400921FB54442D18
decimals=16 eq=1 pi=400921FB54442D18
decimals=17 eq=1 pi=400921FB54442D18
decimals=18 eq=1 pi=400921FB54442D18
decimals=19 eq=1 pi=400921FB54442D18
decimals=20 eq=1 pi=400921FB54442D18
decimals=21 eq=1 pi=400921FB54442D18
decimals=22 eq=1 pi=400921FB54442D18
decimals=23 eq=0 pi=400921FB54442D19
decimals=24 eq=1 pi=400921FB54442D18
decimals=25 eq=1 pi=400921FB54442D18
decimals=26 eq=1 pi=400921FB54442D18
decimals=27 eq=1 pi=400921FB54442D18
decimals=28 eq=0 pi=400921FB54442D19
decimals=29 eq=0 pi=400921FB54442D19
decimals=30 eq=1 pi=400921FB54442D18
decimals=31 eq=0 pi=400921FB54442D19
decimals=32 eq=0 pi=400921FB54442D19
decimals=33 eq=0 pi=400921FB54442D19
decimals=34 eq=0 pi=400921FB54442D19
decimals=35 eq=0 pi=400921FB54442D19
decimals=36 eq=0 pi=400921FB54442D19
decimals=37 eq=0 pi=400921FB54442D19
decimals=38 eq=0 pi=400921FB54442D19
decimals=39 eq=0 pi=400921FB54442D19
decimals=40 eq=0 pi=400921FB54442D19
decimals=41 eq=0 pi=400921FB54442D19
decimals=42 eq=0 pi=400921FB54442D19
decimals=43 eq=0 pi=400921FB54442D19
decimals=44 eq=0 pi=400921FB54442D19
decimals=45 eq=0 pi=400921FB54442D19
decimals=46 eq=0 pi=400921FB54442D19
decimals=47 eq=0 pi=400921FB54442D19
decimals=48 eq=0 pi=400921FB54442D19
decimals=49 eq=0 pi=400921FB54442D19
decimals=50 eq=0 pi=400921FB54442D19

 

PG
Kurt_Bremser
Super User

@mike_jones_SAS Reading slightly different input values to the same stored value because of numeric precision is perfectly understandable.

Reading THE SAME VALUE different just because of the number of insignificant zeros that follow it, and that in only one platform (Windows) that uses the same hardware as another platform that works (Linux x64), and a real-number representation that is common to almost all 64-bit processors today (which also do not display the same error), points to a fault in the Windows code of SAS.

If you find where it happens, and find it is too hard to fix because you would have to basically do MS's work by fixing Windows itself, put that into a message in the SAS knowledge base, so everybody knows it may happen and it is as it is.

 

mike_jones_SAS
SAS Employee

There is an easy workaround for this issue. Any changes to the routine for Windows x64 could not only lead to poorer performance but also has the possibility to introduce other unintended side effects or errors in other computations.

 

SAS Technical Support will pursue a SASnote (or SAS Knowledge Base Message) to document this behavior.

 

ChrisNZ
Tourmaline | Level 20

Glad this is finally recognized as an issue!

 

However, it took several weeks of discussing the problem here (2 threads, 4 pages) and with tech support, receiving shallow replies, spending hours to demonstrate and argue, before something is finally done. And that's just for a Usage Note, not a fix, so the issue remains.


Once again, SAS should eagerly be seeking such feed-back. It should not be so hard and time-consuming. Similarly, there are other issues that were never accepted by tech support for no valid reason and will remain as software quirks/defects/issues.

 

@mike_jones_SAS Please point us to the UN when it is created, I am curious to know what the contents will be. Thanks again for your involvement.

 

 

 

Kurt_Bremser
Super User

It is sometimes funny to watch how big software companies are glued to the status quo, rejecting easily made changes that would get rid of major problems. My pet peeve is the faulty date design that MS copied from Lotus 1-2-3 into Excel and still keeps, although the open source office suites have already shown the correct way to deal with the problem. (For those who don't know it yet: try to enter 29-02-1900 in Excel and in OpenOffice Calc)

 

That the fault actually seems to lead into the depths of Windows itself once again confirms my stance that Windows is the worst possible production platform available today.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 37 replies
  • 2416 views
  • 22 likes
  • 11 in conversation