Re: High Performance procedures in distributed mode

JackHamilton · Posted 10-01-2020 06:33 PM

It seems odd that the High Performance procedures are no longer available for license, especially since version 6 of the manual was published in August, https://documentation.sas.com/api/docsets/prochp/9.4/content/prochp.pdf , but we're told "[my organization] needs to license High Performance Analytics products, which has been retired. HPA functionality has been replaced by Viya, which runs SAS procedures and DATA Step multithreaded and distributed by default." It's OK if you want to install and learn Viya, I guess.

Has anyone made this transition (from running HP procedures in SAS to running them in Viya)? Is it straightforward?

What we're most interested in is not in one of the existing HP procs anyway. We have some PROC PHREG and PROC GENCAT programs that won't run because we can't give them enough memory, and there is no PROC HPPHREG. My impression is that we will have better luck if we can run them in distributed mode under Viya, because each memory usage will be distributed. Is that basically correct?

[Edited to correct formatting that somehow got messed up]

jimbarbour · Posted 10-01-2020 08:56 PM

Jack,

There seems to be quite a bit of emphasis on getting people off of SAS 9 and onto Viya. Discontinuing HP procs in SAS 9.4 seems almost like strong-arm tactics, but I speculate.

Last year, I was working for a company that transitioned from 9.4 to Viya. Performance improvement was anything but automatic. I'm not a SAS administrator or Data Architect, just an analyst and a programmer, but from my limited perspective performance seemed to be very tied to how well the environment was set up and what resources were made available. Viya seems quite a bit more complex in terms of environment and configuration than 9.4, or so I gather.

On some Data steps we were running, we got better performance setting up our own parallel processing using RSUBMITs in SAS 9.4 than we did using Viya, at least we were when I transitioned to where I am working now.

This is from my admittedly limited perspective. My main contribution here may be that posts with more comments tend to get more attention. 🙂

Jim

SASKiwi · Posted 10-02-2020 12:06 AM

Just heard from a SAS-employed software architect that SAS 9.4 M7 is the last maintenance release for SAS 9.4. Customers will need to switch to SAS Viya to get the latest technology and functionality. I had my suspicions but now it is confirmed!

jimbarbour · Posted 10-02-2020 10:39 AM

Sigh. But change is inevitable I suppose, and I can't imagine that SAS would want to staff teams to support changes to both SAS 9 and Viya.

Dr. Goodnight is reputed to have said something about retiring when SAS 10 came out. Perhaps calling it "Viya" just means he plans to stick with it a little longer. 🙂

None of this helps answer @JackHamilton's questions, but at least the post is kept active...

Jim

SASKiwi · Posted 10-02-2020 06:51 PM

@jimbarbour - It is pretty obvious now that SAS 9.x is the legacy platform and Viya is now the all-singing and all-dancing cloud-enabled platform everyone should move to (if not there already). And I don't mean that cynically...

We have yet to make that move but we will do in the next 12 months or so. I too have heard that the switch to Viya is not straightforward. My expectation is that it will take several months of effort to migrate existing applications as some serious modifications will be required. In our case conversion is made harder by having to switch SAS server OS from Windows to linux. Windows Viya has too many limitations apparently to suit our requirement..

JackHamilton · Posted 10-02-2020 07:16 PM

I wonder how many sites will decide to go with a SAS/Viya platform that is just sufficient for ETL, data manipulation, and simple reporting, and start moving advanced statistics to R. From what I hear, new statisticians are learning R, not SAS, and new statistical techniques are arriving in R first. SAS will still have the advantage of procedures that are rigorously tested and systematically verified by a team of highly qualified programmers and statisticians, but that's not important to everyone.

jimbarbour · Posted 10-02-2020 08:10 PM

@JackHamilton,

I think heavily regulated industries (Pharmaceuticals for example) or industries with very significant financial consequences for errors (Insurance, Banking) may still stay with SAS. We'll see. I hope Viya isn't so expensive that it forces companies out of SAS entirely.

Python is the other up and coming language with statistical packages which may well eclipse R (again, we'll see). However, one of the risks of open source is malware. Not every contributor to open source is motivated by altruism. See: https://www.zdnet.com/article/two-malicious-python-libraries-removed-from-pypi/ I'm not aware of malware in R, but the risk is there.

The real issue for veteran practitioners of SAS programming is that SAS has essentially thrown in the towel on the SAS programming language. Whereas before one needed to learn SAS in order to use SAS statistical packages, now R and Python can call Viya statistical packages directly. The value of SAS programming skills is therefore greatly diminished. It's probably a smart move on the part of SAS. Now, graduates coming out of university with either Python or R stat skills can call SAS Viya packages for specialized statistical functions, modeling, etc. without a heavy learning curve or the expense of SAS training. Long term, that's probably the right move for SAS, but not so good for those of us who rely on SAS programming skills to earn a living. So much for my nice little niche.

Of course, there's a lot of code out there, and some industries, see the first paragraph of my reply, will probably remain strong users of SAS. Only time will tell.

Jim

jimbarbour · Posted 10-02-2020 08:12 PM

@SASKiwi,

I will be extremely interested in hearing about how the transition goes for you and what your experience is (if you care to share it). I suppose we'll all be making the transition at some point. Welcome to the brave new world of ~~SAS 9.x~~ Viya.

Jim

mkeintz · Posted 10-04-2020 02:35 PM

@jimbarbour wrote:

Welcome to the brave new world of ~~SAS 9.x~~ Viya.

Yes. Viya con dios

--------------------------
The hash OUTPUT method will overwrite a SAS data set, but not append. That can be costly. Consider voting for Add a HASH object method which would append a hash object to an existing SAS data set

Would enabling PROC SORT to simultaneously output multiple datasets be useful? Then vote for
Allow PROC SORT to output multiple datasets

--------------------------

ximeng · Posted 10-06-2020 11:28 PM

Hi Jack

for your question regarding PHREG and GENCAT...

SAS Viya has a proc for survival data analysis, which can be found here. https://documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.5&docsetId=casstat&docsetTarget=cass...

for PROC GENCAT, do you mean PROC GENMOD?

JackHamilton · Posted 10-07-2020 01:16 AM

Thank you. I have passed that link along to the analyst; she might find it helpful when we get Viya (which is being negotiated).

The question about GENMOD came from an analyst who was working from memory, so she might have misremembered.

ximeng · Posted 10-11-2020 09:27 PM

happy to help.

the two links below might be helpful too. The first one actually contains a comparison of GENSELECT (Viya) and GENMOD.

High Performance procedures in distributed mode