Is changing work directory actually to change the HOME directory?

[fierceanalytics]

Hello, 

I wrote a nice do loop that has 6 layers of macros to test HPFOREST. I know the output, be it log or html, will accumulate to 'endanger' the size limit capped on the HOME/myuserID directory. So I wrote "printto to..." and "ods html path..., file=". 

The long running job,after one night running, is working great. Only that one html file still looms bigger and bigger > 500MB, but another HTML file is on that non-home directory also bigger. My admin emailed me to delete that html. I told him I cannot kill that long job. That model has 7000 variations looking great....

The admin said he cannot relocate my HOME directory. I figured otherwise. I think I did once long time ago, but forgot details. 

Google shows this blog https://blogs.sas.com/content/sgf/2018/05/18/how-to-change-your-working-directory-for-sas-with-the-d....... Changing working directory of SAS, as we did often in Python and R. 

My question: is changing work directory actually to relocate HOME? if not, please how to relocate HOME, links, tips,....?

I think probably we need to go to Management Console with super admin rights. Thanks 

[Reeza]

Do you need the ODS HTML output? Otherwise why not turn it off entirely?
A 500MB file seems unmanageable for many reasons.

I think you need to change the default location using the ideas here:
https://blogs.sas.com/content/sasdummy/2018/08/28/sas-current-directory/

[fierceanalytics]

HTML output critical. I am puzzled
because ods html redirects was on top but hpforest still sent diagnostics to Home. All the other macro, graphics away from home. Does procedure have to go HOME? I don’t see merit forcing that. Thanks

[Reeza]

Are you at least limiting it to what you need? Why not save the output to data sets instead?

[fierceanalytics]

Yes I have saved and computed my differential KS statistics in datasets. I still need html, though. I have cut out those unwanted ods output objects but variables importance.. must be there. If HOME is capped at 1GB, I feel like I am forced to sacrifice analytics. My iPhone has 64GB, it does even care if I fill up 40 GB, but modeling work is enterprise computing. I can split macro to 4 brackets to run sequentially but why? Cutting combination is not very straightforward And I have to keep notes.

[Tom]

Also how are you running SAS?  Are you using some front end, like SAS/Studio or Enterprise Guide.

If you are running regular old SAS (base?foundation?Display Manager?) are you running on a Windows or Unix host?

In general when running a program from the command line the default location for the LOG and LISTING files is the current directory.  You can change that using the -log or -print option on the command line that runs sas.

So example this sequeence of commands will move to you project directory (on some disk with real space allocated to it) and then run the SAS program named myprogram.sas and create the log file as myprogram.log.  Any normal listing output should goto myprogram.lst (or perhaps myprogram.lis if you are somehow still running VAX/VMS operating system).

>cd /myproject_folder
>sas myprogram

[Ksharp]

The following code could check WORK pathname .

proc options option=work value;
run;

[fierceanalytics]

Thank you. This wakes up my old memory of writing lines like "options work=   ;" to redirect. So, no Work is NOT Home. Work is where we process jobs like 'proc sort'. It is by design not to store permanent files. 

So I modify my original question: how to relocate or redirect HOME directory? 

I searched the internet and found many tips on how to relocate Work. I don't have Work directory problem. 

The notion that log, sas, output, html files should be at different directory from data sets is understood. But the default installation size, which appears to be ~1GB?/user in 2020 (since when my HOME hits 400MB my admin called me, several times, and the total size he showed 92% collectively over many users are not very big), is apparently outdated. Now I recalled ~10 years ? I got similar calls for loading too much at HOME. I recall the offending total at that time was ~300MB when we mostly were not using html or ODS output in general. Now some procedures, in my case, HPFOREST, is replicating huge html at HOME while following my redirection of ods html path and proc printto and while the log files were successfully redirected as prescribed. The practical trouble is most enterprise installation follows default setting. 

So, where is the simple 'one-stop-shop' manual on how to relocate HOME directory? Thank you.

[Kurt_Bremser]

To do something about the HOME directory, you need to work with your server administrator.

For one, because he/she's the one with the necessary permissions, and second, if I found a user creating big files where he/she is not supposed to, there's going to be a very substantive talk-to (on the server I am responsible for, it's not possible, anyway; see "one").

I am with you that 1 GB is not much nowadays; our current quota in /home is 3 GB per user. But you see there IS a limit, and it's not THAT large.

I would get in contact with SAS technical support to check what is actually using up all that space. I also question the value of HTML data of this size, as it will cause a problem for the user agent which tries to display that; browsers are already VERY memory-hungry with considerably smaller HTML pages.

And I am with you that there should be a means to redirect that traffic to a place where the limits are less restrictive, like WORK if your quota there is set to a sufficiently large value. And that is also a thing where SAS TS will be your best bet.

[fierceanalytics]

Thank you, Kurt. 

1. No, I don't manage SAS administrators. Needless to say I don't have super, server rights.... 

2. Most, if not all, SAS administrators, my opinion (and you may differ), do not have tradition to reach out to SAS support. 

3. In the case where they do like to work things out, which is this case I have, I need to feed them with sufficient materials for them to know what to ask for when contacting SAS. Most admin staff do not actually know SAS. They approach SAS like it is just another ... 

I will focus on why HPFOREST procedure defies that redirection. I did some test and found only when the procedure reaches certain loops when the HTML starts to mirror-duplicate along side the non-HOME html creation. This is entirely different subject. 

Thank you. 

Jia 

[Tom]

It is not clear to me what your actual problem is from your description.  Its sounds like your program is writing files where you don't have room for them.

I seriously doubt you need to make any changes to your SAS setup.  You just need to fix your program that is calling the HPFOREST to tell it to direct all of its output to a place where there is room.  (or figure out how to have it produce less output).

You could open your own support ticket with SAS support to have them help you understand what files HPFOREST is creating and how to control where it creates them.

[Kurt_Bremser]

---

@fierceanalytics wrote:

2. Most, if not all, SAS administrators, my opinion (and you may differ), do not have tradition to reach out to SAS support. 

---

They need to realize that one of the duties (and I use this word in a very serious manner here!) of a SAS administrator is to be the mediator between their end users and SAS. If a problem is encountered that they cannot solve by themselves (or find a sufficient answer why it can't be solved, so a work around is needed), they have to get into contact with SAS TS. Your organization pays a sh*tload of money for the SAS license, and access to TS is a main part of that, so not using it is a kind of negligence.

Your organization should have a IT helpdesk, so use that to create a ticket for your problem. That is the way problems are handled where I work.

If your pathway to SAS is defective, feel free to contact SAS TS on your own: all you need is your site number, which can be found by running PROC SETINIT, and technical details (SAS version, operating system, ...). You can open a track via web (go to www.sas.com, and from the menu, select Support - Support Services - Technical Support).

If your admins are mainly server administrators that do not feel responsible for anything up from the operating system level, then request that a dedicated SAS administrator is put in place; this might even be one of the end-users (like you) who develops the necessary skills (SAS provides courses for this, and your operating system vendor for the OS - see Maxim 15).

[Nigel_Pain]

I can't add enough likes to your response @Kurt_Bremser. I have been a sysadmin and a SAS admin for over 20 years. I have to say, I couldn't have survived without the help of TS, especially over the last 5-10 years. They are 3rd level support for us, and for any sensibly-minded SAS admin.

[fierceanalytics]

Well, the focus and the root of my post are not really whether SAS TS is useful or not. The key is : if you have 12 TS tickets open today, would you rather to have 8 opened instead? If now you know, by changing package default setting, is going to help reduce the traffic to TS queue. When a TS ticket is open, somebody has already spent time on the issue. And more people will spend more time on it. In 2020, I don't have to tell you opportunity cost of your time, 15-30 minutes if not more.  

Another different subject with the same theme that I ran into, a lot: default memory setting with Enterprise Miner. 8GB RAM? That was 2 years ago. The admin told me it was not possible to change that. Two months later SAS TS walked him through to raise it to 20GB, nice? yes. TS helpful? sure. Only that it past the time I needed it. I had to abandon EM just for that sake. That was not a judgement call on EM or on SAS in general. This is  driving a car: you have to get on whichever road allows you through on the moment for the project, with praise or complaint, or both. 

[Kurt_Bremser]

---

---

@fierceanalytics wrote:

The admin told me it was not possible to change that. Two months later SAS TS walked him through to raise it to 20GB, nice? yes. 

---

---

Such people have a name. It's Mordac.