Numeric format rounds number instead of displaying actual value

Reply
PROC Star
Posts: 2,370

Numeric format rounds number instead of displaying actual value

I recently got annoyed comparing data that showed no difference even when displaying (supposedly) all possible digits .

 

Using long formats was not enough; I had to derive the difference AND use a long format to finally unhide the difference in value.

 

Consider:

data _null_; 
  X=input('409D593333333333',hex16.);
  Y=input('409D593333333334',hex16.); 
  Z=X-Y;
  putlog X 32.20 / Y 32.20 /@1 Z 32.20; 
run;
       1878.30000000000000000000
       1878.30000000000000000000
         -0.00000000000022737368

Am I mistaken thinking that this behaviour is unintended, and that X and Y should be shown with a different value when using a long format?

I know that the difference is the 17th digit, which is beyond numerical precision. That's not the point. We have a difference at the end of the number, we should see it when displaying the whole number.

 

Y is the badly rounded value by the way, but you wouldn't know it by looking at the 3 lines above. You need further computations.

 

Super User
Posts: 6,785

Re: Numeric format rounds number instead of displaying actual value

Just one piece of the pie here ... the 15-digits of precision limit that is often mentioned is for integers.  Decimal fractions may be imprecise with less than 15 digits.

Ask a Question
Discussion stats
  • 1 reply
  • 141 views
  • 0 likes
  • 2 in conversation