topic Re: Scientific Notation in SAS Programming

Scientific Notation

RaniaMamdouh — Thu, 22 May 2014 18:06:04 GMT

Hello all;

How can I convert the following number

5039220013916026846870204011588016812266185989865191407731082739149737501542867149897589000557841

6168937269198370425013463695536280705318595740583841262257709320903463659424481514349542590076222

1723313516671432465293726789169859417220321898386948547503076332140696583782648654162369257874235

6286709130989273095836110164802229755436763344720101376000000000000000000000000000000000000000000

00000000

to the scientific notation 5.039220013916026846870204011588e+395 to use it in the same program in anther statement ?

thanks

Re: Scientific Notation

stat_sas — Thu, 22 May 2014 19:55:40 GMT

Hi,

I don't think this is possible in SAS. As a reference please see the attached link....

Regards,

SAS(R) 9.2 Companion for Windows, Second Edition

Re: Scientific Notation

RichardinOz — Fri, 23 May 2014 00:57:06 GMT

To work with numbers with a very large number of significant digits (as in 5.039220013916026846870204011588e+395) you need to use Fortran or some other application that allows arbitrarily large numbers. SAS, in common with most applications such as MS Excel, uses the operating system's floating point number storage mechanism which in the case of Windows and Unix allows a maximum of 8 bytes, some of which are used for the sign and the exponent. In practice this allows for 15 digits accuracy (16 for integers provided the initial digit is not 9). zOS allows another digit or so because it uses fewer bits for the exponent.

Oracle (apparently) allows 12 bytes to floating point numbers which allows 23 digits accuracy.

You can treat these numbers as character strings, compute the length to determine the exponent, substring the first n characters for the numerical result, and insert a decimal to make it look like the result you want. But you cannot use the result in calculations in SAS. In the format you specify you cannot even import the data in exponential format because the total width of the informat is limited to 32 characters which includes the exponent, decimal point, signs, and 'e'.

In practical terms it is hard to see where a precision of even 32 digits would be required.

Richard

Re: Scientific Notation

FriedEgg — Fri, 23 May 2014 01:32:23 GMT

Let's take a big step out of the box here and solve this...

filename cp temp;
proc groovy classpath=cp;
submit parseonly;

import java.math.BigDecimal;
  import java.text.DecimalFormat;


  class SciNote {
     static String bigSci(String bigint) {
        BigDecimal val = new BigDecimal( bigint + ".0" );
  DecimalFormat df = new DecimalFormat("0.000000000000000000000000000000E0");
return df.format(val).replace("E","e+");
  }     }

endsubmit;
quit;


options set=classpath "%sysfunc(pathname(cp,f))";


data test;
length scinote $ 100;
declare javaobj j('SciNote');
bigint=  "503922001391602684687020401158801681226618598986519140773108273914973750154286714989758900"
||       "055784161689372691983704250134636955362807053185957405838412622577093209034636594244815143"
||       "495425900762221723313516671432465293726789169859417220321898386948547503076332140696583782"
||       "648654162369257874235628670913098927309583611016480222975543676334472010137600000000000000"
||       "000000000000000000000000000000000000"
;
j.callStaticStringMethod('bigSci',bigint,scinote);
put scinote=;
run;

From the LOG:

scinote=5.039220013916026846870204011588e+395

Re: Scientific Notation

FriedEgg — Fri, 23 May 2014 01:35:08 GMT

SAS can work with additional datatypes such as bigint, float, etc... within the bounds of PROC DS2, but the general sentiment of your statements is accurate. If you are going to be working consistently within this context, SAS is not currently the appropriate place.

Re: Scientific Notation

RichardinOz — Fri, 23 May 2014 04:12:10 GMT

@FriedEgg

I'm very grateful to you for drawing my attention to Proc DS2, which is a very recent addition to SAS. I looked at the definition of Float which implies Double allows precision of over 25 significant digits (though the definition is hazy on this point, and the word 'approximate' in the definition is a bit of a worry). So 32 digit precision might be possible within Proc DS2 and Proc FedSQL. But Bigint is limited to 19 digit precision, so I doubt the OP could use these techniques on his data.

My question still is, what is the OP doing with with such large numbers?

The age of the universe is currently reckoned at 13.8 billion years or 4.36e20 seconds.

The observable universe diameter is 8.948e26 meters and even converting that to Angstrom or Planck units will not require 100 significant digits.

The number of atoms in the observable universe is estimated to be between 1e79 and 1e83, counting the total number of particles would only add a couple of orders of magnitude.

Richard

Re: Scientific Notation

FriedEgg — Fri, 23 May 2014 15:30:30 GMT

Cryptography is one of areas where numbers of this size are prevalent, to give you a real-world example as you are seeking. Of course, I cannot speak to the OP's purpose.

RSA-2048 = 2519590847565789349402718324004839857142928212620403202777713783604366202070

7595556264018525880784406918290641249515082189298559149176184502808489120072

8449926873928072877767359714183472702618963750149718246911650776133798590957

0009733045974880842840179742910064245869181719511874612151517265463228221686

9987549182422433637259085141865462043576798423387184774447920739934236584823

8242811981638150106748104516603773060562016196762561338441436038339044149526

3443219011465754445417842402092461651572335077870774981712577246796292638635

6373289912154831438167899885040445364023527381951378636564391212010397122822

120720357

Re: Scientific Notation

RaniaMamdouh — Sat, 24 May 2014 11:28:09 GMT

When I run the program, the following massage was appeared:

filename cp temp;

proc groovy classpath=cp;

ERROR: Procedure GROOVY not found.

submit parseonly;

------

ERROR: A link must be established by executing the SIGNON command before you can communicate with

PARSEONLY.

WARNING 14-169: Assuming the symbol RSUBMIT was misspelled as submit.

ERROR: Remote submit to PARSEONLY cancelled.

what can I do?

anther question can I convert the number to the scientific notation in Proc iml???

Thanks for your help

Re: Scientific Notation

FriedEgg — Sat, 24 May 2014 14:37:59 GMT

PROC GROOVY is new in 9.3, so you would require that version or newer to run it. Alternatively, you can still run my example by first creating the Java class outside of SAS. You can find an example I wrote on a different subject for how to go about doing it:

As for using IML, in SAS/IML Studio the IMLPlus language will allow you to run Java code, which you could use as I've illustrated to perform your operation.

You could also process it manually as a array of integers, which is I believe how the BigInteger class in Java works with these numbers.

Re: Scientific Notation

RaniaMamdouh — Sat, 24 May 2014 17:29:22 GMT

How can I process it manually as a array of integers???

Note that I use SAS 9.2

Thanks

Re: Scientific Notation

Rick_SAS — Mon, 26 May 2014 11:24:03 GMT

The "big question" (ha ha) is what you intend to do with this number. Factor it? Divide by it. Unless you are a researcher in number theory, numbers of this size usually indicate that you should change to a log scale. Statisticians use quantities like log-likelihood and log-determinants because numerical computations that overflow on the original scale succeed on a log scale. Number of this size also arise in combinatorial computations when you try to compute a formula that includes factorials. There are better ways to compute these quantities, so why don't you provide a little background about what you are tryng to accomplish and how this number came up?

Re: Scientific Notation

RaniaMamdouh — Mon, 26 May 2014 17:34:12 GMT

Hi Mr. Rick;

At first I would like to thank you for your cooperation with me.

Here I tried to compute the values of "xi" to get it's roots,but when i computed the "xi" I need to compute a big factorial and here the large number appeared and the program stop because of the storage space. So i need to convert the large number to scientific notation to reduce the space.

proc iml;

start BigFactorial(n);

start LogFact(n);

return( sum(log(1:n)) );

finish;

numDigits = ceil(logfact(n)/log(10));

f = j(1,numDigits,0);

f[1] = 1;

do i = 2 to n;

carry = 0;

digits = ceil(logfact(i)/log(10));

do j = 1 to digits ;

g = i*f + carry;

digit = mod(g, 10);

f = digit;

carry = (g - digit)/10;

end;

return( f[,ncol(f):1] );

finish;

i={10 9 8 7 6 5 4 3 2 1 0};

do j=1 to 11;

m=100;

n=5;

n=int(n);

alpha=((m*(n-1))-2)/2;

f = BigFactorial(10+alpha);

Fact10alpha = rowcat(char(f,1));

f = BigFactorial(alpha+i);

Factalphaij = rowcat(char(f,1));

xi= (Fact10alpha*((-1)**i))/(fact(10-i)*Factalphaij*fact(i));

xivector= xivector//xi;

end;

r = polyroot(xivector);

print alpha;

print xivector[format=E32.14];

print r;

quit;

Re: Scientific Notation

FriedEgg — Mon, 26 May 2014 17:51:17 GMT

What is the purpose of expressing the value in scientific notation?

Re: Scientific Notation

RaniaMamdouh — Tue, 27 May 2014 09:07:39 GMT

When I run the program with this large number the program stop for the following error:

xi= (Fact10alpha*((-1)**i))/(fact(10-i)*Factalphaij*fact(i));

xivector= xivector//xi;

end;

ERROR: (execution) Character argument should be numeric.

operation : * at line 1692 column 17

operands : FACT10ALPHA, _TEM1003

FACT10ALPHA 1 row 1 col (character, size 396)

503922001391602684687020401158801681226618598986519140773108273914973750154286714989758900055784161689

372691983704250134636955362807053185957405838412622577093209034636594244815143495425900762221723313516

671432465293726789169859417220321898386948547503076332140696583782648654162369257874235628670913098927

309583611016480222975543676334472010137600000000000000000000000000000000000000000000000000

_TEM1003 1 row 1 col (numeric)

statement : ASSIGN at line 1692 column 1

but when I used the scientific notation this error doesn't appear.

Re: Scientific Notation

Kurt_Bremser — Tue, 27 May 2014 12:17:47 GMT

"Scientific Notation" is just a format through which a SAS number (with all its 8-byte limitations) is displayed.

fact10alpha is not numeric, but a character variable that CANNOT be used as a number value in a computation, because with such a big string the automatic conversion (that SAS usually tries) fails.

Re: Scientific Notation

RaniaMamdouh — Tue, 27 May 2014 12:49:07 GMT

How can I convert the fact10alpha to a numeric to use it in the computations???

Re: Scientific Notation

Kurt_Bremser — Tue, 27 May 2014 12:53:18 GMT

That is the core of the problem: you cannot. SAS simply can not deal with numbers that fall outside of what can be handled with 8 bytes real.

Get the right tool for this kind of math and do it there. SAS is not the right tool.

Re: Scientific Notation

RichardinOz — Tue, 27 May 2014 12:54:13 GMT

To deal with numbers of the order of 1e395 as an integer without loss of precision you would need a machine representation of around 165 bytes (against SAS use of floating point 8 bytes - which does not allow even 2**64 precision). Using scientific notation does not change this requirement.

The example using Proc Groovy demonstrates there are other tools (Java, in this instance - I showed my age in mentioning Fortran) which can accommodate very large numbers. You should use those tools.

Any attempt to carry out numerical analysis on this scale in SAS is an exercise in futility.

Richard

Re: Scientific Notation

Rick_SAS — Tue, 27 May 2014 13:52:51 GMT

Again, the question is "what are you trying to accomplish." It looks like you are trying to fnd the roots of a tenth-degree polynomial with coefficients given by some alternating expression that involves ... binomial coefficients? Looks like there is an "n choose k" term, or in your notation, "alpha choose i," but there are other terms as well.

What is the polynomial, and why are you trying to find its roots?

By the way, I wouldn't trust the result of the POLYROOT function on a polynomial with such huge coefficients. How do I know that the coefficients are huge? Well, you can compute the coefficients without computing factorials of the large numbers. Note that

(alpha + p)! = alpha! * (alpha+1) * (alpha+2) * ... * (alpha + p)

so that the term (alpha + 10)! in the numerator cancels most of the terms of (alpha + i)! in the denominator.

You are left with a numerator that is the simple product: (alpha + i + 1) * ... * (alpha + 10).

The denominator simplifies to (10 + 1)! * i!

From these computations, the coefficients of the i_th term of the polynomial are roughly of the order +/-alpha##(10-i). You can use the PROD function to compute the coefficients exactly, but again I emphasize that this is NOT something I recommend. Instead, I recommend understanding the purpose of finding the roots.

Re: Scientific Notation

RaniaMamdouh — Tue, 27 May 2014 18:07:54 GMT

I try to find the roots to calculate the nodes and weights of the Gauss-Laguerre integration.

Do you mean that

xi= (Fact10alpha*((-1)**i))/(fact(10-i)*Factalphaij*fact(i));

will be

xi=numerator/ denominator

numerator= (alpha+i+1)* (alpha+1)* (alpha+2)* (alpha+3)* (alpha+4)* (alpha+5)* (alpha+6)* (alpha+7)* (alpha+8)* (alpha+9)* (alpha+10)* ((-1)**i)

denominator=(fact(10-i)*fact(10+1)*fact(i))