kevind Tracker

Re: Setting COMPRESS=YES as default on Linux Grid servers

kevind — Tue, 21 Mar 2017 11:53:57 GMT

Earlier the datasets I did the COMPRESS tests on had a high level of compression but I ran COMPRESS on a dataset with a lot of binary data and the output COMPRESSed dataset became 90G instead of 72G and it took quite a while longer to output it than the uncompressed dataset. I had seen earlier in a paper that SAS would not compress a dataset that would be bigger than the uncompressed but when I went back and looked at the paper, it was quite old (SAS 8). I'm using SAS 9.4 M3.

Re: Setting COMPRESS=YES as default on Linux Grid servers

kevind — Mon, 20 Mar 2017 19:50:31 GMT

Thanks for mentioning that but yes the CLEANWORK utility runs hourly on each server via crontab. Also on a daily basis will kill any SAS bjobs that are older than 5 days which has helped a lot.

Re: Setting COMPRESS=YES as default on Linux Grid servers

kevind — Mon, 20 Mar 2017 12:24:58 GMT

Hi Juan

Regarding the question about getting users to add the OPTIONS COMPRESS=YES, what I mean is that I can send out an email to all users but only a portion of them will take action which is why I was interested in making the compress=yes a default.

I was wondering why the SASWork and Utilloc usage would go up? I was hoping that the COMPRESS option might free up room there too by compressing WORK datasets as well. Our WORK volume gets heavily used for large amounts of data since WORK is a 10TB flash drive and much faster than DASD and has more free space available most of the time than some teams have on their DASD volumes. Several times a week there are multiple users using terabytes of WORK storage and I have to kill the jobs of the largest consumer to avoid being 100%.

Setting COMPRESS=YES as default on Linux Grid servers

kevind — Fri, 17 Mar 2017 13:38:59 GMT

Does anyone have the COMPRESS=YES as the default for their SAS users? We still have available CPU cycles but we get very low on our very pricey disk space and its not possible to get all users to put the OPTIONS COMPRESS=YES in their code. Are there any concerns with doing this?

Re: Enterprise Guide 7.1 fails to submit grid job

kevind — Fri, 24 Feb 2017 18:05:29 GMT

I resolved by bouncing the ObjectSpawners. I knew this would be the resolution but it's something I'm reluctant to do with 140 connections running. Only one user was having the issue and I first tried removing and re-adding them with SASMC. I'm maintaining a SAS 9.4M3 Grid.

Re: The Copy Files task is an official task now

kevind — Mon, 20 Feb 2017 18:30:21 GMT

I think this is great news for me as an admin because our security team is going to start blocking SSH port 22 that is used for SFTP. I'm assuming the copy task uses a SAS port and if SAS/Secure is enabled, that the transmission is encrypted. Do you know if this is the case?

SAS for (Docker) Containers

kevind — Thu, 09 Feb 2017 21:31:38 GMT

Anyone deploying SAS for Containers to be used by multiple users in multiple departments? I'm new to the cloud and have it running but I'm working thru how to add users, attach persistent disks, transfer data thru buckets, provide command line access in addition to the SAS Studio, automate shutdown and restart of the VM with the dockers, switch machines monthly, have separate dockers running for each user but have all users connect on same port 80.

Re: How can I log EG file transfers for auditing

kevind — Fri, 02 Dec 2016 16:06:29 GMT

I realize that there will be other ways for content to be downloaded off of the server (copy/paste, email, etc) but I've started on this journey to write the tools to know what's occurring on the system and I'll capture these other items as they get identified. I used the logconfig.trace.xml on WS and I didn't see the IOM::FileService events but I did see evidence of the file transfer in these messages:
2016-12-01T15:22:23,078 TRACE [00002894] 2:sas - Bridge PE [7fad0c395fd0] 7fad000a3aa0: 00 00 00 00 2f 68 6f 6d 65 2f 73 61 73 00 01 00 |..../home/sas...|
2016-12-01T15:22:23,078 TRACE [00002894] 2:sas - Bridge PE [7fad0c395fd0] 7fad000a3ab0: 0d 00 00 00 73 61 73 63 68 65 63 6b 2e 6c 6f 67 |....sascheck.log|
Do you have any examples of capturing the IOM:FileServices?

BTW, I recently changed WS logconfig.xml to record errors and wrote a Perl script that reformats the log data into a CSV file and emails it to me so that I now have visibility to users having database connection errors instead of waiting for a SAS user to report the issue and guess how many others are having the issue.

Re: How can I log EG file transfers for auditing

kevind — Wed, 30 Nov 2016 16:28:40 GMT

I found that the EG logging on the PC does capture the copying but I need to track it on the server and haven't found where that would be logged or what service on the server is providing the transfer. Thanks for the feedback.

2016-11-30 10:55:18,614 [17] INFO SAS.Tasks.CopyFiles.SasFileTransferTask [(null)] - Running Copy Files task: Copy Files

,,,

2016-11-30 10:57:21,448 [17] INFO SAS.Tasks.CopyFiles.SasFileTransferTask [(null)] - Checking for existence of target folder: C:/users/kcd01/Downloads
2016-11-30 10:57:21,448 [17] INFO SAS.Tasks.CopyFiles.SasFileTransferTask [(null)] - DOWNLOADING files...

Re: How can I log EG file transfers for auditing

kevind — Tue, 29 Nov 2016 15:31:46 GMT

We are using SAS 9.4 M2 Grid on RHEL 6.8. So far I have just enabled WS logging on QA but I want to do so for production so that we can detect SAS/ACCESS errors. I feel like I can manage the logs (scan them for errors and remove them) but I don't want to have any unnecessary performance hits. I configured the logging with ImmediateFlush false. So far using threshold INFO but assuming that the ERROR threshold would be enough to detect the ACCESS issues I'm looking for. Regarding the EG file copy, this will be a really good feature, assuming it's encrypted, since Security wants the ssh sftp shutdown, but they would also want some way to review what files are uploaded/downloaded from the servers.

How can I log EG file transfers for auditing

kevind — Tue, 29 Nov 2016 14:46:15 GMT

I'd like to have audit logs of files transferred between the server and the PC by Enterprise Guide users using the Copy Files task. I currently have the Workspace logconfig.xml configured to record job logs but there is nothing being logged regarding file transfers. Any ideas?

Re: What admins topics would you like to see covered during SAS Global Forum 2017?

kevind — Wed, 12 Oct 2016 18:34:44 GMT

Hi Kurt, last May I converted a SAS 9.4m2 Redhat Grid over to authenticating with Active Directory using sssd/pam services. Along with that I had to change ownership and groups for 700,000+ files. It's much better for us now that we don't have users maintaining local accounts on multiple Linux servers. I can share some of the details with you next week if you think it would be useful for you.

Re: What admins topics would you like to see covered during SAS Global Forum 2017?

kevind — Tue, 11 Oct 2016 17:12:07 GMT

Another topic for me would be along the lines of Site Reliability Engineering, what tasks should an admin automate to maintain, update and check on health of their systems. Currently we are on 9.4 M2 but we are looking to 9.4 M3 to have the new version of Environment Manager to better manage SAS.

Re: What admins topics would you like to see covered during SAS Global Forum 2017?

kevind — Tue, 11 Oct 2016 17:04:13 GMT

I'm interested in hearing about successes/failures of implementing SAS in the cloud (preferably Google Cloud) and how to decide which approach to take (containers, SAS 9.4 on VMs, or Viya).

Re: Is EMiner 14.1 client compatiable with EMiner 13.2 server?

kevind — Wed, 01 Jun 2016 13:08:05 GMT

Thanks for the insight. I should have mentioned that the server is a RHEL 6.x 2 node grid. It's a pretty big effort to update our servers (with 250+ SAS users) from 9.4 M2 to M3 but the client PCs will be upgraded from Win 7 to 10 this summer. It's nice that we can take the EGuide from M3 and push that out. I'm now trying to determine what needs to be done for EMiner (only 5 users) since we aren't planning to upgrade the servers until Q4.

Is EMiner 14.1 client compatiable with EMiner 13.2 server?

kevind — Thu, 26 May 2016 19:49:06 GMT

Just wondering if a newer EMiner client is backwards compatiable with an older version on the server the way EGuide is usually backwards compatiable. Also, do I need to upgrade to EMiner 14.1 if using Windows 10?

Re: Using RSA SecurID with SAS Enterprise Applications

kevind — Thu, 28 May 2015 19:53:52 GMT

Well, it's three years later and I'm on SAS 9.4m2 two node Grid on RHEL 6.x and needing to do some security hardening with the RSA SecurID. Is RSA working for you and do you have any tips about installing and configuring for EGuide and EMiner?

Re: CPU pinning for SAS processes?

kevind — Fri, 21 Nov 2014 13:00:23 GMT

Jaap, Thank you for pointing me in the right direction! Found a SAS paper too support.sas.com/resources/papers/proceedings14/SAS289-2014.pdf. This is exciting stuff.

Re: CPU pinning for SAS processes?

kevind — Thu, 20 Nov 2014 20:39:23 GMT

I have RHEL 6.5 but the Cgroups functionality is not installed. Have you set up any clients with this?

CPU pinning for SAS processes?

kevind — Thu, 20 Nov 2014 18:19:19 GMT

Here's the Scenario: 2 servers (Redhat) with 8 cores each but 4 cores are turned off on each server to be in compliance with SAS licensing. So the issue is, that the 4 cores on each server that are running SAS applications are also competing with 700 other processes, for monitoring, clustered file system, backups, scheduling, file transfers, etc.

I found the post below about using the "taskset" command to reserve cores during boot up and then start specific applications on those cores. Has anyone used this function or one like it to reserve cores for SAS (Grid) and then use the remaining cores for the rest of the processes?

How to run program or process on specific CPU cores on Linux
Last updated on October 29, 2013 Authored by Dan Nanni 11 Comments

As multi-core CPUs become increasingly popular on server-grade hardware as well as end-user desktop PCs or laptops, there have been growing efforts in the community (e.g., in terms of programming models, compiler or operating system support) towards developing applications optimized for multi-core architecture.

One operating system (OS) support often exploited to run performance-critical applications on multi-core processors is so-called "processor
affinity" or "CPU pinning". This is an OS-specific feature that "binds" a running process or program to particular CPU core(s).

<cut>

kevind Tracker

Re: Setting COMPRESS=YES as default on Linux Grid servers

Re: Setting COMPRESS=YES as default on Linux Grid servers

Re: Setting COMPRESS=YES as default on Linux Grid servers

Setting COMPRESS=YES as default on Linux Grid servers

Re: Enterprise Guide 7.1 fails to submit grid job

Re: The Copy Files task is an *official* task now

SAS for (Docker) Containers

Re: How can I log EG file transfers for auditing

Re: How can I log EG file transfers for auditing

Re: How can I log EG file transfers for auditing

How can I log EG file transfers for auditing

Re: What admins topics would you like to see covered during SAS Global Forum 2017?

Re: What admins topics would you like to see covered during SAS Global Forum 2017?

Re: What admins topics would you like to see covered during SAS Global Forum 2017?

Re: Is EMiner 14.1 client compatiable with EMiner 13.2 server?

Is EMiner 14.1 client compatiable with EMiner 13.2 server?

Re: Using RSA SecurID with SAS Enterprise Applications

Re: CPU pinning for SAS processes?

Re: CPU pinning for SAS processes?

CPU pinning for SAS processes?

Re: The Copy Files task is an official task now