Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Home
- /
- Programming
- /
- Programming
- /
- Compute large number in SAS

Options

- RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Posted 09-04-2018 09:29 AM
(2546 views)

14 REPLIES 14

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

@Rick_SAS has any idea ?

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

What types of calculations are you trying to do? Can you just change the scale of the numbers?

In addition to the magnitude issue there is limit to the number of significant digits that can be represented.

188 data _null_; 189 length constant $20 value 8 ; 190 do constant='EXACTINT','BIG','SMALL' ; 191 value =constant(constant); 192 put constant value best32.; 193 end; 194 run; EXACTINT 9007199254740992 BIG 1.7976931348623E308 SMALL 2.2250738585072E-308

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

When numerical analysts need to deal with large numbers, they use the log scale. For example, you should never compute the likelihood function, which will usually overflow, but always use the log-likelihood function. Similarly, never compute the determinant of an ill-conditioned matrix when the log-determinant is what you need.

SAS has many built-in functions such as LOGPDF that compute the logarithm of standard probability distributions, or LGAMMA, LCOMB, and LFACT, which produce the log of various combinatorial numbers. See my blog post on this topic.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Hi, I need to compute this equation (with a, d, u, eb, es, b, s are all given). I just need to plug in the number, but SAS could not compute it if (u or b or s) is large

I think this equation can not be transformed further by using LOG or other transformation

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Please provide the values of the parameters.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Hi @Rick_SAS. Some typical cases in my sample that SAS can not compute are:

a=0.58 d=0.41 u=344.17 eb=470.89 es=388.29 b=1415 s=104 or

a=0.18 d=0.09 u=983.15 eb=184.69 es=280.79 b=1097 s=760

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

What problem are you trying to solve? Is this a likelihood function? If so, for what distribution? How did you obtain the parameter estimates?

When I see large numbers like these, I always try to ascertain where the numbers came from and what they will be used for. Suppose I tell you that the value 1/pi is 4E187. What does this imply? For example, if pi is a probability, it would mean that the probability is effectively zero. Does that extreme value make sense for your application, or are you expecting a reasonable probability like 0.1?

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Thanks @Rick_SAS. Yes, you are correct, Pi here is probability. And if I know that 1/pi is large. For example, if 1/pi is 4E187, meaning that the probability is effectively zero. Then I could set Pi = 0 (I don't need the extremely low value of Pi).

The issue is that there is no way I can tell whether Pi is extremely low (so that I could comfortably set it to zero) or Pi has a meaningful value unless I can compute the equation above. But whenever u>709, or B or S large enough, SAS can not compute these intermediate values: e^u or (1+u/eb)^b. So I can't tell whether in this case, Pi is extremely low (in this case it has no meaning), or it will take meaningful values like 0,1111.. or 0.234516 ....

So this is the problem. Do you have any idea of how i can get around this issue?

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Hi @Rick_SAS . The parameters are estimated using maximum likelihood method (EKOP 1996, page 11). I am confident that I get these parameters correctly.

Then these parameters are plug into the equation, which I obtained from a recent paper in a very high quality journal below (in page 28)

http://www.acsu.buffalo.edu/~swhuh/y_daily_prob_Appendix.pdf

The authors mention about how to overcome the overflow issue. I followed them but there are still many cases with large numbers that can not be computed.

So do you know whether there is any method to overcome this (maybe another softwares/machines?)

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

*> I would want to know which is the largest (integer) y in (1+x)^y that SAS can compute?*

You can do this:

```
data _null_;
do X=1 to 10;
Y=int( log(constant('big')) / log(X+1) );
put X= Y= ;
end;
run;
```

X=1 Y=1024

X=2 Y=646

X=3 Y=512

X=4 Y=441

X=5 Y=396

X=6 Y=364

X=7 Y=341

X=8 Y=323

X=9 Y=308

X=10 Y=296

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Actually, You can;t use the numbers in my previous reply. This yields useful numbers:

```
data _null_;
do X=1 to 10;
Y=int( log(2**1023.999999999) / log(X+1) );
put X= Y= ;
end;
run;
```

X=1 Y=1023

X=2 Y=646

X=3 Y=511

X=4 Y=441

X=5 Y=396

X=6 Y=364

X=7 Y=341

X=8 Y=323

X=9 Y=308

X=10 Y=296

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Thank you. But anyone knows how to compute these large number or get around this issue.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

I glanced at the references that you provided. It seems that the paper maximizes the likelihood function (Eqn 16, p 1414) instead of the log-likelihood. Switch the log-likelhood and the overflow will probably disappear,

If you don't take this advice, then other options include:

1. You can do extended precision computations in languages such as Mathematica.

2. To get SAS to compute the answer you would need to rewrite the expression. In the appendix of the paper that you linked to, they do that and reduce the number of times that overflows occur (p. 4, Eqn 17) and discussion therein.

3. Provided that e1 = mu/eps_B and e2 = mu/Eps_S are less than 1, you can estimate those terms by using the binomial expansion of the expressions (1 + e1)^B and (1 + e2)^S. However, the number of terms that you choose to keep depends on the smallness of e1 and e2, which are data dependent, so this idea might not work for your data.

4. Sometimes the parameters become more manageable if you center or standardize the data. For example, if you measure volume in units of "hundred million shares" or share prices in units of "ten dollars," that could make a difference in the magnitude of the parameter estimates.

Good luck!

**Available on demand!**

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.