PROC MCMC: Same Seed producing different results between two SAS versions?

[Digicha1]

Hello,

This is probably an obvious answer, but does Base SAS 9.4 M6 and PC-SAS 9.4 TS Level 1M7 produce different random number generations in PROC MCMC even if the same seed is set?

I'm not able to replicate the posterior datasets, but the posterior summaries are close to 2 decimal places at least.

Is there a way I can set an older version of SAS in 1M7 in order to replicate?

Thank you.

[ballardw]

Pretty much the only way to be sure of generating the same random number sequences is on the same version of SAS on the same operating system and may require the same hardware (actual CPU ). Algorithms get tweeked in SAS and operating systems. 

Some options may allow you to access to specific hardware options with the TRNG option in Streaminit that might get closer to the same exact sequences but I wouldn't hold my breath.

[Digicha1]

Thanks for your reply!

It's a shame this is an issue, as it makes reproducibility such a pain! I'll see if I can get the same SAS versions to see if this gets closer estimate at least.

[WarrenKuhfeld · Accepted solution]

@ballardw Gave a good reply. If you compute something simple, say a simple regression analysis, you would rightly expect the results to be the same across machines and SAS releases--at least to the number of decimal places that you would see in output.

That all changes when you get into complicated iterative algorithms, and algorithms with random number seeds is one example. It could be the case that a procedure might not change but a new math library was installed (or some other underlying thing changes), and some more accurate calculation causes the results to no longer exactly match. I saw this over and over when I was involved with experimental design algorithms when I was a SAS developer.

[SteveDenham]

Following up on Warren's statement, I always try to  do some sensitivity analysis when a method depends on a random number - especially for Bayesian analyses. Running with 10 to 20 seeds and examining results and traces is pretty informative. Sometimes the data is so ill behaved that none of the results match within an acceptable range.

SteveDenham

[WarrenKuhfeld]

@SteveDenham Good advice!