topic Re: int function odd behavior in SAS Programming

int function odd behavior

SP_SAS — Wed, 08 Jul 2020 04:30:48 GMT

Hi Friends,

I ran below code

data y;
cnt = 2.55;
a = cnt-int(cnt);
d = (a>0.550);
e = (a=0.550);
f = (a < 0.550);
run;

f comes out as 1, which indicates that sas is considering 0.55 as less then 0.55. Any thoughts

Re: int function odd behavior

ketpt42 — Wed, 08 Jul 2020 04:46:45 GMT

Here's the documentation of the INT function: https://documentation.sas.com/?docsetId=lefunctionsref&docsetTarget=p1b1zd0wuyufp0n1jbqlr27p4eay.htm&docsetVersion=9.4&locale=en

The result is fuzzed, so this is expected behavior. Using INTZ will produce the results you are expecting.

data y;
cnt = 2.55;
a = cnt-intz(cnt);
d = (a>0.550);
e = (a=0.550);
f = (a < 0.550);
run;

Re: int function odd behavior

Tom — Wed, 08 Jul 2020 05:01:51 GMT

Welcome to the world of storing decimal numbers using binary numbers. 0.55 and 2.55 cannot be exactly represented by binary numbers.

data y;
cnt = 2.55;
a = cnt-int(cnt);
diff = (0.55-a);
d = (a>0.550);
e = (a=0.550);
f = (a < 0.550);
run;

proc print; run;

Obs     cnt      a        diff       d    e    f

 1     2.55    0.55    2.2204E-16    0    0    1

Re: int function odd behavior

Kurt_Bremser — Wed, 08 Jul 2020 06:55:36 GMT

It has nothing to do with the INT() function, but with the way binary math works in calculations:

data _null_;
x1 = 2.55 - 2;
x2 = (x1 = 0.55);
put x1= x2=;
run;

Log:

73         data _null_;
 74         x1 = 2.55 - 2;
 75         x2 = (x1 = 0.55);
 76         put x1= x2=;
 77         run;
 
 x1=0.55 x2=0

Re: int function odd behavior

SP_SAS — Wed, 08 Jul 2020 14:32:57 GMT

Here is the best answer for my purpose.. let me know if you guys think anything can go wrong with this solution.

data y;
cnt = 2.550;
intaa = int(cnt);
intzz = intz(cnt);
a = cat('0.',scan(cnt,2,'.')) *1;
d = (a>0.550);
e = (a=0.550);
f = (a < 0.550);
run;

Re: int function odd behavior

SP_SAS — Wed, 08 Jul 2020 14:35:23 GMT

same problem with intz too.

Re: int function odd behavior

PaigeMiller — Wed, 08 Jul 2020 14:45:30 GMT

@ketpt42 wrote:

Here's the documentation of the INT function: https://documentation.sas.com/?docsetId=lefunctionsref&docsetTarget=p1b1zd0wuyufp0n1jbqlr27p4eay.htm&docsetVersion=9.4&locale=en

The result is fuzzed, so this is expected behavior. Using INTZ will produce the results you are expecting.
data y;
cnt = 2.55;
a = cnt-intz(cnt);
d = (a>0.550);
e = (a=0.550);
f = (a < 0.550);
run;

I don't see how INTZ helps here. Also, on my computer, when I run the above code, I get f=1.

Re: int function odd behavior

PaigeMiller — Wed, 08 Jul 2020 14:49:03 GMT

@SP_SAS wrote:

Here is the best answer for my purpose.. let me know if you guys think anything can go wrong with this solution.

data y;
cnt = 2.550;
intaa = int(cnt);
intzz = intz(cnt);
a = cat('0.',scan(cnt,2,'.')) *1;
d = (a>0.550);
e = (a=0.550);
f = (a < 0.550);
run;

I think things can go wrong. Here's code that I think I can trust:

data y;
cnt = 2.550;
a = cnt-int(cnt);

d = fuzz(a-0.550)>0;
e = fuzz(a-0.550)=0;
f = fuzz(a-0.550)<0;
run;

Re: int function odd behavior

Tom — Wed, 08 Jul 2020 14:48:23 GMT

You are not addressing the actual issue. If you only want to compare to 4 decimal places then round the numbers. Might want to round to 4 decimal digits and then test.

want = round(have,0.0001);

See this paper: https://www.pharmasug.org/proceedings/2014/CC/PharmaSUG-2014-CC50.pdf

Re: int function odd behavior

Tom — Wed, 08 Jul 2020 14:58:16 GMT

The MOD() seems to fix this. Not sure if it does for all values or just this specific one.

The real solution is to ROUND() the value(s) before comparing.

542   data y;
543     cnt = 2.55;
544     a = cnt-int(cnt);
545     b = mod(cnt,1);
546     c = round(a,0.0001);
547     a_test = (a=0.550);
548     b_test = (b=0.550);
549     c_test = (c=0.550);
550     put (_all_) (=/);
551   run;


cnt=2.55
a=0.55
b=0.55
c=0.55
a_test=0
b_test=1
c_test=1

Re: int function odd behavior

PaigeMiller — Wed, 08 Jul 2020 15:05:20 GMT

The real solution is to ROUND() the value(s) before comparing.

I'm not convinced. Numbers rounded as above to 4 digits cannot be represented exactly in binary. So couldn't there be cases where we're back to square one, where the binary representation causes the problem at the start of this thread?

Re: int function odd behavior

Tom — Wed, 08 Jul 2020 15:11:38 GMT

I don't think SAS has an almost equal operator. So you need to round, at some point, before comparing. You could include it in the code that does the comparison.

if round(a/b,0.0001) = round(c/d,0.0001) then do;

@PaigeMiller wrote:

The real solution is to ROUND() the value(s) before comparing.

I'm not convinced. Numbers rounded as above to 4 digits cannot be represented exactly in binary. So couldn't there be cases where we're back to square one, where the binary representation causes the problem at the start of this thread?

Re: int function odd behavior

Rick_SAS — Fri, 10 Jul 2020 09:37:41 GMT

I'm late to this discussion, but a guiding principle for numerical computations is "don't perform exact comparisons on floating-point numbers." You can use the COMPFUZZ function in Base SAS to help compare two values for equality.

For inequalities, the typical fuzzy comparison is to compare a difference with a "small value." "Small" is relative to the magnitude of the numbers. For example: [EDIT: I had signs wrong originally]

Instead of testing x < y, you can test (x - y) < -epsilon
Instead of x > y, you can test (x - y) > epsilon

The article "Avoid testing floating-point values for equality" includes references and some discussion of how to choose epsilon.

Re: int function odd behavior

PaigeMiller — Wed, 08 Jul 2020 15:37:44 GMT

@Tom wrote:

I don't think SAS has an almost equal operator. So you need to round, at some point, before comparing. You could include it in the code that does the comparison.
if round(a/b,0.0001) = round(c/d,0.0001) then do;

I quibble with your statement that you "need to round". There are other choices. The FUZZ function, as I pointed out earlier, or the COMPFUZZ function (as pointed out by @Rick_SAS) seem to address the issue, and both functions are indeed, in layman's terms, an "almost equal operator".

And I'm still not convinced that round is a solution for all possible floating point numbers.

Re: int function odd behavior

Kurt_Bremser — Wed, 08 Jul 2020 17:18:12 GMT

@PaigeMiller wrote:

And I'm still not convinced that round is a solution for all possible floating point numbers.

If both values in the comparison are rounded to the same digit, and they differ by a sufficiently small epsilon, the resulting binary "error" will be identical, so the comparison will work.

Re: int function odd behavior

Rick_SAS — Wed, 08 Jul 2020 18:50:30 GMT

Kurt, I haven't seen explicit code from you, so perhaps I am misunderstanding, but I think in principle Paige is correct. If the two numbers are on opposite sides of a cutpoint, the rounding will take them in opposite directions.

data A;
epsilon = 1e-16;
x = 0.00005;
y = x - epsilon;
RndX = round(x, 0.0001);
RndY = round(y, 0.0001);
put RndX=;
put RndY=;
run;

Re: int function odd behavior

FreelanceReinh — Wed, 08 Jul 2020 19:45:14 GMT

Hi @SP_SAS,

Here is what happens internally in the calculation 2.55-2:

In the binary system (mathematically), the decimal number 2.55 looks like this:

10.1000110011001100110011001100110011001100110011001100110011001100...

The pattern "0011" is infinitely repeating.

The internal 64-bit floating-point representations of 2.55, 2.55−2 and 0.55 in SAS under Windows (as documented in Numerical Accuracy in SAS Software) can be displayed with the BINARY64. format:

720  data _null_;
721  do x=2.55, 2.55-2, 0.55;
722    put x binary64.;
723  end;
724  run;

0100000000000100011001100110011001100110011001100110011001100110
0011111111100001100110011001100110011001100110011001100110011000
0011111111100001100110011001100110011001100110011001100110011010

You see the repeating pattern here as well. The binary digits highlighted in blue represent the fractional parts of the numbers. For 2.55 two more bits (red and purple) are needed due to the non-zero integer part: These are the two digits surrounding the point (".") in the mathematical binary representation above. When 2 is subtracted from 2.55, the integer part cancels out, these two bits are removed, the subsequent bits are shifted to the left and leave room for two bits at the end.

In the result 2.55−2 these two least significant bits are simply filled with zeros (highlighted in bold) because the representation of 2.55 doesn't contain the corresponding information. The best possible representation of 0.55, however, contains non-trivial digits in these places: The exact last two digits 01, followed by 10011001... in the mathematical representation, are rounded up to 10.

So, the difference between 0.55 and 2.55−2 in the internal representation is the "1" in the penultimate bit. This "1" stands for the value 2**-52=2.2204...E-16 (see @Tom's variable diff).

Re: int function odd behavior

FreelanceReinh — Wed, 08 Jul 2020 19:52:24 GMT

@Tom wrote:

The MOD() seems to fix this. Not sure if it does for all values or just this specific one.

The real solution is to ROUND() the value(s) before comparing.

Thanks for pointing this out. It's interesting that the MOD function seems to handle a number of these cases correctly, albeit in general only up to four decimals, e.g., mod(2.12345,1) ne 0.12345, whereas equality holds for decimals .0000 through .9999.

I guess this is a result of the "extra computations, called fuzzing, to return an exact zero when the result would otherwise differ from zero because of numerical error" (documentation). The remarkable thing is that this involves creating correct, non-trivial binary digits of the result which are not contained in the argument, especially if the integer part is large.

Example:

data _null_;
x=1414.2135;
y=mod(x,1);
z=.2135;
put x= binary64. / (y z)(+13 =binary64. /);
run;

Result:

x=0100000010010110000110001101101010011111101111100111011011001001
             y=0011111111001011010100111111011111001110110110010001011010000111
             z=0011111111001011010100111111011111001110110110010001011010000111

I've indented the (equal) values of y and z in order to align the parts of the mantissas which represent the fractional parts of the three numbers: For y and z, these are the 39 blue and the 13 green bits. The latter are not contained in the internal representation of x because 10 bits (red) are occupied by the integer part (1414) and 3 additional bits (purple) are needed for the fractional part (without normalization). Moreover, the least significant bit is rounded up.

I virtually always use the ROUND function (typically with rounding units such as 1e-9) when comparing numeric values with a small to moderate number of decimals.

Re: int function odd behavior

ChrisNZ — Thu, 09 Jul 2020 03:47:36 GMT

@PaigeMiller >And I'm still not convinced that round is a solution for all possible floating point numbers.

My experience is that rounding always works if done appropriately. Since the error is always very small, rounding by say a billionth gets rid of it.

To reuse @Rick_SAS 's code:

data A;
  X = 0.00005;
  Y = 0.00005 - 1e-16; 
  A=X=Y; 

  RndX = round(X, 1e-9);
  RndY = round(Y, 1e-9);

  B=RndX=RndY; 
  C=RndY=0.00005;

  putlog A= B= C=;
run;

A=0 B=1 C=1

Though I agree than in principle the decimal number cannot be represented, at least rounding correctly homogenises the binary values.

Re: int function odd behavior

Kurt_Bremser — Thu, 09 Jul 2020 04:28:19 GMT

@Rick_SAS wrote:

Kurt, I haven't seen explicit code from you, so perhaps I am misunderstanding, but I think in principle Paige is correct. If the two numbers are on opposite sides of a cutpoint, the rounding will take them in opposite directions.
data A;
epsilon = 1e-16;
x = 0.00005;
y = x - epsilon;
RndX = round(x, 0.0001);
RndY = round(y, 0.0001);
put RndX=;
put RndY=;
run;

You are right, that's an edge case where round() can't fix it.