How do I use Git with my SAS projects? Q&A and webinar recording
Did you miss our live webinar "How Do I Use Git With My SAS® Projects?"? No problem! You can watch the recording right now. [Last updated Dec 2022]
In this article you'll find a summary of the session, attached copy of the slides presented, and an extended Q&A wrap up. The webinar features a number of interactive demos, so be sure to view the recording.
For a quick tutorial, check out this entry on the SAS Users YouTube channel.
About this session
Few technologies have done more to advance code collaboration and automation than Git. And GitHub’s explosive growth has drawn the interest of all types of programmers, including SAS programmers.
Many SAS products, including SAS Enterprise Guide, SAS Studio and the SAS programming language, now have direct integration with Git – and that extends to GitHub.
- Learn many ways that you can now add source control, code collaboration, and continuous integration and deployment to your SAS processes.
- See examples of how SAS experts share their content on GitHub.
- I've updated this article with the latest slides and Q&A from December 2022.
Q&A from the webinar
Can you share the links to other resources you talked about in the webinar?
- How to connect SAS to your Git profile
- Workshop: Using Git with SAS (on GitHub)
- Git functions in SAS 9.4 and SAS Viya (doc)
- How to organize your SAS projects in Git
- Using Git in SAS Enterprise Guide (doc)
- Git with SAS Studio and SAS Enterprise Guide (video)
- SAS Software on GitHub
- Using built-in Git operations in SAS (blog)
- DevOps with SAS 9: SAS code, GitLab, and Jenkins (community)
- developer.sas.com for SAS app development
- Pro Git by Scott Chacon and Ben Straub, free online book about Git
- SAS Note about generating SSH key for use with Git and SAS
I primarily write and update SAS programs on my own. Does it make sense to use Git?
You can get a lot of benefit even as a sole developer for using Git. I have a lot of projects that I manage -- SAS code projects. We have an internal GitLab repository where I I put all of these. It helps me to stay organized. It helps me to have that version control. But it's also good for continuity of business. I may be biased, but I think these code projects are important assets to my company. Having these in Git repos ensure others can pick them up in my absence.
The system I use doesn't give me access to the command line. Could I still use the SAS Git functions?
The Git functions within SAS do not require that you have access to the command line or that you be allowed to run operating system commands directly. See an example of using Git functions to initialize a repo, stage files, commit and push -- all within a SAS session.
What would be the difference in using Git that is already built-in to SAS EG? Will it require any coding on a Git side?
The Git feature that's built into SAS Enterprise Guide works on EG projects only - using embedded SAS programs. It's fine for a single user who wants to take advantage of program history and restore. For true collaboration and working with typical dev practices, you can use Git to manage programs outside of your project -- which reside in the file system, and you simply reference these as shortcuts in your project. There is nothing you need to install for EG to work with this, but you would need an external Git repository (collection of files managed in Git). Typically that's going to come from your organization's Git environment or a service like GitHub.
Is there a programmatic way to view any tags that have been added to a commit?
I think the current answer is no, not from within SAS. But the Git functions receive regular enhancements, so it could be added someday.
How do you ensure SAS server program files are kept up to date with the latest code in the remote repo (assuming the remote repo is synced with a local repo on the user's local machine)?
When pushing files you need to make sure that the upstream repository or the remote repository always has the latest versions of changes and your local repo must be in sync with these before you can push. The way Git operates, you won't be able to push content in a "stale" repo without first pulling the latest and reconciling any conflicts.
Are there plans already to add SAS Studio flows into Git Versioning?
Great question - it is not supported today but I could see it happening. Check out the vast library of SAS Studio custom steps available on github.com/sassoftware!
Our group has a bunch of shared macros we often reuse across different projects. Can we use Git in SAS to add the shared macros to a new repository we clone?
Yes, that's a great use of Git. Imagine a process that pulls in the latest macro library and adds to to the autocall search path!
options dlcreatedir;
%let repopath=%sysfunc(getoption(WORK))/dept-macros;
libname repo "&repopath.";
data _null_;
rc = gitfn_clone(
"https://github.com/company/dept-macros",
"&repoPath."
);
put 'Git repo cloned ' rc=;
run;
/* add these macros to the autocall search */
options append=(sasautos="&repopath.");
Is there a way to make Git employ file locking? The lack of file locking is sad.
The best practice of "rebasing" your code repository before committing/pushing usually takes care of conflicts you might have when teams are working in the same code. That's the "Git" way of managing this. That said, some services do offer a file-locking feature in Git (GitLab Premium is an example).
Which version of SAS Enterprise Guide was the first to support Git?
Version 7.1, as highlighted here. Best integration is with SAS Enterprise Guide 8.2 or later.
Does SAS Studio have a similar integration with Git as SAS Enterprise Guide?
SAS Studio allows you to manage Git repositories with external code in the same way that SAS Enterprise Guide does. Feature-wise, these are almost the same, yes.
The repos that you are showing in EG, they sit on a remote server rather than on someone's client?
They may originate from a remote server, but the clone of Git repo (my working version) is on the desktop client (with EG).
Similar to Git functions in Base SAS, the Git repo is on the client? If it is on the client, do we need to install Git or does the SAS Base client come with Git in-built?
You don't need to install any Git tools to use the Git functions with Base SAS. For Git functions, the Git repo (working files) must be in the file system shared by your SAS session. Typically that's a remote machine when using SAS Studio, but could be local if you are using Base SAS on Windows.
Are you guys deploy applications on cloud providers like AWS, Azure or Google Cloud?
Yes! In fact, SAS has a major partnership agreement with Microsoft working with Azure -- but all cloud providers are also supported.
For using SAS Data Integration Studio, does that allow for different versions of deployed jobs and flows?
See this article for all of the details about Git with SAS Data Integration Studio.
For SAS Enterprise Guide, can Git authentication can route through Okta?
Yes - this is single-signon. You would use HTTPS authentication, which requires a username and password. In place of an account password, use an access token. Your Git service (example, GitLab) can be used to configure an access token). Here's the slide where I discussed this in the webinar:
Are there any functionality that is not supported in SAS Studio but available in Git Bash?
Git Bash provides access to all of the Git commands -- and there are so many with so many options. SAS Studio doesn't cover them all, especially with the many ways you have to manage merging and branching and packaging commits. But it does offer most of the common tasks, including stash, push, pull, fetch, commit, add (stage), and others.
Is there a difference for Git between SAS code and Python code?
I don't think so. Git excels at managing code assets, regardless of the programming language.
Will Git work with stored compiled macros?
Compiled SAS macros are catalog files in SAS. These are binary and dependent on architecture (operation system, bitness). You can manage the files in Git, but you won't get the benefit of code history, diffs, etc.
Does integrating Git violate any of [my company's] cyber policies reading data from online sources?
Most organizations have standards around code assets and management in Git (or some source management). Your organization might have an internal-only Git environment that they direct you to use. It's very possible/likely that they have a policy that governs how you use external sites such as GitHub to fetch code or store code.
Thank you for doing this talk! I find it really useful to check git logs for specific files (e.g. on command line "git log [filename]"). I could do this in EG 8.1 but can't in 8.2. Any advice?
For this I like the GIT_COMMIT functions:
/* Clone to a temp space -- I don't even need to know where */
options dlcreatedir;
%let repoPath = %sysfunc(getoption(WORK))/sas-dummy-blog;
libname repo "&repoPath.";
libname repo clear;
/* Fetch latest code from GitHub */
data _null_;
rc = gitfn_clone(
"https://github.com/sascommunities/sas-dummy-blog/",
"&repoPath.");
put rc=;
run;
data _null_;
n = gitfn_commit_log("&repoPath.");
length message $ 50;
message = "";
rc = gitfn_commit_get(1,"&repoPath.",'message',message);
put n " commits, most recent commit: '" message "'";
run; Is there a platform for unit testing SAS code?
Check out SASUnit - might be interesting to you.
Could you describe high-level steps to upload existing SAS project to new GitHub repo?
These are the same as for any new code project -- here's a list of the Git commands you might use.
- git init (to create a repo)
- git add (to add source files to the repo)
- git commit -m "[commit message"] for your first commit to the repo
- git remote add origin 'your-repo-url-in-git-server' - to associate your repo with a new upstream location.
- git push (to move your new project to the Git server)
There are other ways to do this without Git commands, using tools like Git GUI or the web browser with whatever Git service you're using.
Have you ever faced with using Git + SAS in Clinical Trials? Do you know any problems in using Git + SAS in that area?
In such a regulated industry I'd expect source management -- with its ability to produce audit trails -- to be very important. However, I don't know what the industry standards are. Great question for the Life Sciences Analytics Framework team at SAS!
How can we config Git on SAN file system on SAS Enterprise Guide 8.2?
If your file path is a network path (\\unc style or mapped drive) available from EG, there should be no special set up. Simply clone your Git repo to that location and add to EG's list of repositories.
VS Code = Visual Studio Code?
Yes. It's gaining a growing number of users who manage SAS code along with other code.
My company uses GitHub Enterprise. Can I use the SAS GIT* functions in my programs to work in that environment?
Yes, you can. The GIT* functions in Base SAS use the standard Git library for integration, and so should work with any Git-based system that adheres to that protocol -- including GitHub Enterprise, Gitlab, and Bitbucket.
I currently use PROC HTTP and GitHub APIs for Git integration. Is there an advantage to using the GIT functions instead?
PROC HTTP is good way to use the GitHub APIs (see an example here). But for source management operations, I think you'll find that the Git functions are more natural, require less code, and less overhead around issues like authentication.
Will SAS Enterprise Guide add the ability to clone from remote repositories, and push changes back to the origin repo?
Yes, this has been added into SAS Enterprise Guide v8.2. Here's a screenshot:
What benefits do you see that Git has over Subversion (SVN)?
All the cool kids are using Git. Even at SAS, where we have large investments in more established (and some proprietary) source control systems, we're using Git for almost all new projects. The workflow is flexible and there are many developer-oriented tools that integrate with it. Still, if all you have is Subversion, it's better to use that and add rigor around your SAS processes than to forgo source control altogether. Most SCMs offer integration at the file system level, and you can leverage that for your SAS programs. Here's a blog post that describes how this can work.
Which EG version(s) have Git built-in?
Program history (and Git integration in general) was added in SAS Enterprise Guide 7.1. (In the webinar I said v7.13 -- incorrect. It was available in the initial v7.1, in the field since Oct 2014.)
For the Git support on internal EGP files - does this support multiuser concurrent access?
No, it does not. EGP files do not support multiuser access in general. If you use external, referenced SAS programs in your project, then Yes, of course you can collaborate with others simultaneously.
Where is the commit message and/or where is that info provided if you click "Commit" button?
When you click the Commit button, you'll be prompted to add a commit message.
The commit message is added as part of the commit operation in Git, whether you commit via the button in EG or via an external tool.
If we're committed to using GitHub would you recommend that we also use the Git internal to SAS EG? Or should a developer track granular changes with Git for Windows or the GitHub server?
No, I recommend using externally referenced SAS programs as much as possible -- in this case. This provides you with the ability to use more tools to manage your SAS content, as really only SAS Enterprise Guide can read/write project files (EGP files).
Is the Pull option in SAS Studio really just a fetch? or is it actually fetching/merging against your local repo?
No, I think it's using Git pull (using this function under the covers). There are arguments/opinions about whether "fetch and merge" is better than just "pull" (which also merges, but might not give you the control you need). Still, I think where you see Pull in the user interface, it equates to the Git pull command.
I'm not clear on the differences between local repository and external repository. Can you please explain?
I used several terms in webinar to characterize repositories. In SAS Enterprise Guide, you can track program history in embedded programs via the internal repository that EG tracks inside the project file. That's not available to external Git tools. You can also work with files in a local repository -- your working copy of a Git repo that tracks all of your changes, additions, etc. And then you can sync that local repo with a remote repo, in a central server like GitHub or your enterprise Git.
If that doesn't clear it up, here's a helpful diagram that I lifted from this Git Tutorial from edureka.
Can the Base SAS functions work with both local & remote repositories?
Yes, the functions allow you to clone remote to a local, and push your local changes back to remote. You do need to set up the authentication with SSH public/private keys. See more doc on possible workflows in the SAS documentation.
We use Git in Visual Studio at our Office. Do we have an Desktop tool for it to push content to online repository just like GitHub desktop?
Is this what you use (link to Microsoft's doc about Git and VS)? It's just another Git client as far as I can tell, so it should work with your SAS content (if you manage it there), whether connecting to public GitHub or a private enterprise Git.
Are there plans to integrate the ability to push and pull with SAS EG in later versions? What if you upgrade your SAS installation to 9.4 M6 -- will you be able to use the base SAS functions within EG?
Yes, that's in the roadmap -- to extend EG's Git integration. But if you have SAS 9.4M6, you can use those Git functions within code that you run from SAS Enterprise Guide. My demo in the webinar included some of that.
If I have a Git repository set up with a group of SAS programs, can I open it up in EG? What changes will EG make to the repository?
First, you need to clone the repository to a local file location on Windows (where EG is running). And then EG will recognize that those files are managed by Git (as you open them in EG), and you'll have access to Commit and History functions. To sync back out with the origin repository, you'll use an external tool (Git command, Git GUI, GitHub Desktop). Update: in SAS Enterprise Guide v8.2, you can use EG to manage the entire process, clone to push.
Did I hear correctly that the new Git commands available for EG rely on prerequisite of also having SAS Studio installed? My SAS license is for only Base SAS+EG, so does this mean that I cannot leverage this feature?
No, the Git functions are used by SAS Studio, but they don't require SAS Studio. They are part of Base SAS as of SAS 9.4M6.
If we're using something like Travis CI, how would we install SAS into the container to test our SAS code?
SAS provides rapid deployment "recipes" for various patterns of using SAS in the cloud. SAS also has offerings on the AWS Marketplace as well as Microsoft Azure. Some of these are purely for SAS Viya, but SAS for Containers can support SAS v9. You (or your IT team) would need to provision SAS in a cloud that Travis CI could reach.
I'd like to see how MS Teams can use Git integration with SAS. And can you explain how trigger notifications work with Slack?
Both Slack and Microsoft Teams offer the ability to connect external apps/services to their channels like GitHub. Here's the one for Slack, and here's Microsoft Teams. Each work in a similar way, allowing you "subscribe" the channel to notifications of events from GitHub.
In addition, Slack and MS Teams offer APIs that allow you to create custom notifications based on whatever rules you want. You can even call these APIs from SAS code. Here's an example of how to do this for Slack.
Are the source code files saved to a MS cloud? Can the remote repository be an internal location?
No, Microsoft cloud (or Office 365) has nothing to do with this -- unless you count the fact that Microsoft (the company) owns GitHub. And yes, your Git repository can be completely internal to your organization. You can use an internally hosted Gitlab, GitHub Enterprise, or even just a plain old Git server.
Is there any external tool to connect to GitHub?
Just about any Git client can connect to GitHub -- GitHub Desktop, Git command line, Git GUI, Git integration with Visual Studio, etc.
Will the SAS Git functions be included in SAS Viya?
Yes, these are planned for the next major release of SAS Viya, currently planned for late this year.
What is Gitlab used for?
Gitlab is a Git-based tool, like GitHub or Bitbucket, that offers value-add integrations on top of the Git source control infrastructure. Companies usually pick one of these offerings as a standard to use...but sometimes an org might dabble in multiples of these.
As a developer, do you find it easier to automate processes in Python or another language that calls SAS?
I use SAS to automate almost everything...that I automate. Python is great, but SAS is just as capable (more so for data-driven applications) and obviously a favored tool around here.
A bit off-topic, but what's the status for Enterprise Guide 8.1? And will it have even deeper Git-integration?
The status of SAS Enterprise Guide 8.1 is AWESOME. Oh, and available soon (mid 2019). Deeper Git integration is on the roadmap for a later release.
Are only the SAS programs/macros added to the Git repo?
SAS Enterprise Guide manages just SAS programs in Git (for commit/changes/history). SAS Studio can recognize other content in your repo, I think. And the SAS functions can work with any file that you choose/specify in the functions.
My organization is using 9.4 Maint 5. Is there a way I might make use of the Git functions, like GITFN_DIFF, without upgrading to 9.4 Maint 6?
No, not in this same way. If you try it, you might find that SAS 9.4M5 "knows" about the function, but doesn't actually work. Some of the hooks were added in M5 but the Git supporting library didn't ship until M6 (when the feature was completely tested and supported).
Can you talk a little bit more about forking?
Sure. "Forking" in GitHub is the act of making a copy of one person's/organization's repository and pulling it into your own account. You can then make whatever changes you want without affecting the original. If you want to contribute some of those changes back to the original, you use a pull request (PR) that puts your proposed changes into a review queue. The repository maintainer can then accept those changes, reject them, or suggest further changes. Some pull requests can go through many iterations. For example, this PR on saspy has gone through lots of work and validation, and I think it's very close (at the time of this writing) to being accepted/merged.
Are there any big watch for items (gotchas) to be careful of?
Yes, Game of Thrones spoilers. I have to be very careful to avoid certain news outlets - even the headlines are giving too much information away!
But you probably mean about Git (not #GoT). I think on balance, you're safer adopting some sort of version control practice instead of none. So if Git is your first foray into this, you're about to take a big leap forward in your coding/backup practices.
Does Git handle metadata charges in DI? Or do you use another tool for that?
Version Control in SAS Data Integration Studio works at the object level, allowing you to archive/version jobs and other objects. Using the comparison tools in SAS DI Studio, you can compare content between versions of these objects. Read more in the documentation. And read more about the Git integration in this article.
What about your example repository that you used in the demo? Can you share that and the code you used?
Sure, if you're super interested about my family's Netflix streaming habits, you can find the repository of code, data, and SAS Enterprise Guide project here: https://github.com/cjdinger/sas-netflix-git. Here's the code you can run from a SAS environment that has SAS 9.4 Maint 6, showing how to use those GIT* functions.
/* Clone to a temp space -- I don't even need to know where */
options dlcreatedir;
%let repoPath = %sysfunc(getoption(WORK))/sas-netflix-git;
libname repo "&repoPath.";
libname repo clear;
/* Fetch latest code from GitHub */
data _null_;
rc = gitfn_clone(
"https://github.com/cjdinger/sas-netflix-git/",
"&repoPath.");
put rc=;
run;
options source2;
/* run the code in this session */
%let _SASPROGRAMFILE=&repoPath./code/import_netflix_activity.sas;
%include "&repoPath./code/import_netflix_activity.sas";
%include "&repoPath./code/find_duplicates.sas";
%include "&repoPath./code/netflix_report.sas";
@GenDemo With the latest changes in SAS Enterprise Guide v8.2, I think you have all of the support you need for a solid Git-based workflow with integrated Git tools, even if you use tools like Gerrit or SmartGIT. Check the topics in the "About Git integration" section in the SAS Enterprise Guide help.
This works best with program files that are managed outside of a project -- that is, not embedded into the EGP. And with SAS Enterprise Guide v8.1 and later, the application supports "project-less" work, so you can use the tool as an IDE -- even if you don't want to leverage project flows. Of course, projects are still fully supported.
