Viya 4 doesn't use .sas files?

[Quentin]

High level, meandering, intro question about Viya.

For my whole career of SAS programming, I've used .sas files to store SAS code. That's been on PC, linux server, SAS 9 BI server, etc.  (I can't claim VAX, but I've inherited .sas files where the header indicates they files started off running on VAX).

I had assumed that in Viya, a programmer would still have .sas files sitting somewhere in an OS directory on a Viya server.  In a SAS 9 BI environment I use EG, and write stored processes and DI jobs, but they all %include .sas files sitting in a linux directory on the SAS server.  Which means I can use whatever tools I want to manage the .sas files, since they're just text files.

I learned yesterday that in Viya 4 SAS code is not stored in .sas files (this statement may be wrong).  Instead the SAS code is stored in a postgres database, which I guess is the successor to the SAS 9 BI metadata server.  So you can access your code through Studio or other SAS tools, but you wouldn't be able to open a file using a text editor.  Because there is no text file to open.

I think the explanation was that in Viya 4 because the Viya server is in a container, that container is replaced with every update, so there is no place for a user to haver persistent file storage.

Is that right?

What does that mean for example for the %include statement in Viya 4.  I see there is one, and it says you can %include an external file which the OS knows about :

'external-file'
specifies the physical name of an external file that is enclosed in quotation marks. The physical name is the name by which the operating environment recognizes the file.

That suggests a Viya SAS session can see files on an OS.

But I see from an old question about Viya (https://communities.sas.com/t5/SAS-Studio/include-statement-in-VIYA/td-p/603099) that the Viya user was storing their code in Viya rather than on the OS, so the answer was to use the filename FILESRVC access method to access code stored in Viya files service.

Actually looking again at that old answer, I can see that the %include does still refer to a .sas file.  That's a relief.

So maybe the Viya 4 change is about where .sas files are stored.  In SAS 9, .sas files are stored in OS directories.  In Viya 4, .sas files are stored in the the "Viya files service"  (Infrastructure Data Server?).  Is that right?

And related, in SAS 9 if I use ODS to write a PDF report, I write that to an OS directory for persistent storage.  In Viya, would I write the PDF file to the Viya files service?

Or would I maybe have a linux server attached to the Viya environment that could be used for persistent OS file storage (perhaps for PDF files or .sas files).

Can I write any type of file to the Viya files service?  How easy is it to get non-sas files out of the Viya files service (Is there a Viya files services GUI client for browsing files, analogous to WinSCP for browsing linux dirs)?

Sorry for rambling questions, just trying to start to wrap my head around this a bit.  Happy for links to any helpful blog posts/UG papers that describe this aspect of Viya.

[SASKiwi]

I'm not an expert on this but from what I understand you can create persistent storage for use with SAS Studio on Viya 4. See this doc link.

You can also interface SAS Studio with Git to use that to manage your SAS code files. AFAIK SAS programs remain as text files with a SAS extension.

[Quentin]

Thanks @SASKiwi that's good to see.  Hopefully Studio will be something like EG on our 9.4 server, where I have an EG project, but the code in the project is just a link to a file on the OS.  The fact that you can set up persistent storage for Studio explains why the docs say %include can include a file from the OS.

I'm still curious about whether Viya jobs and other Viya 'objects' can see .sas files stored in persistent storage on an OS.

On our 9.4 BI server, I can make a DI Studio job (which exists in Metadata), but the only code in metadata is a %include statement which %includes a .sas file sitting on the OS.  Similarly I can make a stored process for web reports, and the only code in the stored process is a %include.  That way all my .sas code can be maintained with whatever I tools I want, I can use the OS for moving files around, diffing files, version control, etc. And availability of persistent file storage on the OS means one macro autocall library stored on the OS can hold macros used by a stored process, a DI job, an EG session, or even PC SAS.

It looks like the Viya SAS Job Execution Web App is the successor to SAS 9 Stored Process Web App.  https://documentation.sas.com/doc/en/pgmsascdc/v_030/jobexecug/n055josnxfatfwn1pyr7p1ah7225.htm And the jobs run on the Compute Server, which is like old fashioned SAS9 rather than CAS.  But I can't tell if a job running on the Compute Server would have access to persistent file storage anywhere.  The flow diagram in the overview only shows access to the file services provided by Viya.

On BI 9, when you make a stored process in EG or SMC, you have the option of pasting all your code for the stored process into a dialog box and storing it all in Metadata.  That always seemed like a nightmare to me, because I like EG and other tools for editing SAS code.  So I'm hoping Viya will allow a similar way to limit the amount of code stored in file services  /postgres database rather than as .sas files on the OS.

But again, good to see that Studio clearly can connect to persistent storage.  That's a relief.

[SASKiwi]

@Quentin  - You can use the SAS Content Assessment tool to convert your EG projects to SAS Studio process flows. SAS Studio Engineer replaces SAS DI Studio in SAS 9. BTW, the licensing model for Viya 4 is radically different to SAS 9. SAS is strongly pushing Viya as Software As A Service (SAAS) and it appears many SAS customers are heading down that path. DIY is also available but the complexity of Viya 4 makes SAAS a more attractive option.

[Quentin]

The beauty of having .sas files is I won't even need to migrate my EG projects.  Because my EG project files are literally just a collection of links to .sas programs.  I don't actually use EG process flows, or other EG stuff that would generate code or control code.  Same for my DI jobs, I can just recreate them in whatever new app.  As long as that app allows me to have a simple job with a single user-written code node "%include mycode.sas".  I tend to avoid permanent macro catalogs, format catalogs, template itemstores, etc for the same reason.  For most of my work, the benefits of having it all in code outweigh the efficiency gains of pre-compiling that stuff.

Yes, I can absolutely see the appeal of SAAS, with a Viya environment hosted by SAS.  I'm not confident that with sensitive healthcare data, we'd be able to get that approved (or if it was approved, it might drastically limit the internal databases we could connect to).  So if we can't do SAAS, I think the next best thing would be for us to host Viya on-prem (or in our cloud tenant or whatever), and then pay SAS consulting team to be the admins.  They said they support both options, which is great.

I would feel much better bout having SAS administer the environment.  Even for SAS 9 BI server, as a user, I feel like SAS Administration has been a challenge for many organizations.  You either take a linux admin and force them to learn about SAS (which they seem not to love), or you take a SAS person and teach them how to be an admin.  I'm sure there are companies that have figured it out. But I like that SAS is offering remote admin services to those who can benefit from it, in addition to full SAAS.

[andreas_lds]

---

@SASKiwi wrote:

I'm not an expert on this but from what I understand you can create persistent storage for use with SAS Studio on Viya 4. See this doc link.

You can also interface SAS Studio with Git to use that to manage your SAS code files. AFAIK SAS programs remain as text files with a SAS extension.

---

Right now i have to use certain tools to manage my code (and documentation) due to company rules. Would be a pity to say goodbye to sas because our tool-set won't be usable any more. Pessimists calculated, that we will need more than two years to migrate everything: tons of SPs with really fancy stuff like starting sas sub-processes on our app-server so that parts of jobs run in the background ... And, of course, 90% of our sas code is stored on a file server and we won't change that, being forced to have a sas licence to read code files is, simply put, not acceptable.

[AllanBowe]

If I were you I wouldn't care too much about how or where Viya stores your SAS programs.  If your source code is in GIT, you can clone it and manage your files using any IDE on any operating system.  SAS Studio is one option, using git functions, or - for a more realistic 'developer experience' - VS Codium.

You don't have to look very far outside SAS to see that this is the dominant paradigm for code management in most languages.  Clone a repo, make a branch, lint & deploy & test changes, make a pull request, and after it is merged, the new version is deployed.

Viya should not be considered more than a simple 'runtime' environment, IMO, and defnitely not the 'source of truth' when it comes to your SAS programs.  That should always be the main branch of your GIT repository.

There are many tools for handling SAS code outside of a SAS runtime, with the benefit that it you can choose your SAS flavour - be that Viya, SAS 9 EBI, or Base SAS.

[Quentin]

Understood Allan, and can certainly see many benefits of the full DevOps approach to SAS application development.

But note that for SAS data scientists / analysts etc who are writing code but are not paid to develop software and are not in software development groups, this might be less feasible.

GIT of course is good for everybody, including data scientists and analysts.  Indeed, if I can have all my .sas code as files on the file server, and GIT is running on the same server, this makes basic use of GIT easier, right?  Or at least gives me more options.  I've heard of folks who have a  /DEV  OS directory of SAS code and a /PROD OS directory, sitting on a SAS 9 server, and the .sas files are the source code for stored processes.  When they need to make an update, they pull the code from GIT repo into /DEV, do the work, test it, then push it up to the repo, then go to /PROD and pull down the code.   Done.  So it's obviously more manual, but for a small shop of one or two people, I can see how it's useful.  And the only tool they need in addition to SAS is git.  This works easily because all the code is not stored inside a stored process stored in metadata.  It's just files, so as soon as the files are updated by the git pull, the stored process has been updated.

For the worse non-git approach, people will simply promote .sas files from /DEV to /PROD with a cp command.

Of course not suggesting this approach is better than having automated approach for build/test/release.  Just saying that a SAS environment that has the option of using .sas files gives more flexibility to SAS community (even at the cost perhaps of less automation, slower performance, etc.)

[Quentin]

Also, the broader questions about Viya SAS session access to a file server with persistent storage isn't only about .sas files.  It's also about output files (e.g. PDF report generated by a SAS job), files with control data that may be maintained by non-sas processes, etc etc.