Learning SAS? Welcome to the exclusive online community for all SAS learners.

Anybody else have a problem with truncated log files using SAS UE batch submit?

Accepted Solution Solved
Reply
Frequent Contributor
Posts: 130
Accepted Solution

Anybody else have a problem with truncated log files using SAS UE batch submit?

Having finally solved many problems with some research analysis in SAS UE I was looking forward to not babysitting code execution so much by using the batch submit feature. Execution was error free as expected, The approx 6MByte html output file was produced correctly but the 40KByte log file was truncated at around line 660 - with less than another 100 lines to record until sucessful termination. Is this a fixed SAS UE batch setting? I've searched Studio batch preferences but log file size is not mentioned. During normal interactive submission all the log is visible. At the time of the mysterious truncation, proc iml is running, having been invoked from Warren H Kuhfeld's wondeful %choiceff experimental design macro.


Accepted Solutions
Solution
‎11-25-2016 09:37 PM
Frequent Contributor
Posts: 130

Re: Anybody else have a problem with truncated log files using SAS UE batch submit?

I found it under the 'more application options' tool button immediately to the left of 'help' tool button, in the left hand side of the studio tool bar. The tool button is the one that looks like 3 horizontally stacked books with an elipsis ... underneath. Select Preferences... . Under the General tab there is a Time-out interval setting which you can increment or decrement in 0.1 hour units using up or down arrow buttons.

 

The locked-down states of SAS UE studio and SAS OnDemand Studio differ in how big you can make this timeout, which makes sense. My CPU time is my own, whereas SAS OD's CPU time is generously donated to students by SAS! SAS OD Studio timeout setting stops at a maximum of 3.0 hours, whereas SAS UE Studio timeout goes way beyond that. I will leave it as an exercise for the reader to see just how stupidly large one can set it in SAS UE.

 

View solution in original post


All Replies
Regular Contributor
Posts: 231

Re: Anybody else have a problem with truncated log files using SAS UE batch submit?

Hey Damien!

 

I hope you're doing well :-)

 

I have tried a number of different Tasks, code chunks, etc. and cannot find anything that runs the log that many lines.  Guess your code is a little more hardcore than anything I can come up with watching TV on the evening of Labour Day!

 

Having said that, I'm wondering if the Log / Results / everything are limited by the space allocated by the VM.  I know there are settings in Virtual Box that can be tweaked (cautiously, as it can apparently break your VM).  I'm thinking that a 6MB results file plus code and everything caused the log to force-truncate, as the VM literally ran out of room.

 

Be interesting if @AmyP_sas , @BeverlyBrown or @ChrisHemedinger have any further thoughts about settings that can be tweaked?

 

Good luck and please post back if you figure it out - i'm curous where this ends up!

Chris

Has my article or post helped? Please mark as Solution or Like the article!
SAS Super FREQ
Posts: 361

Re: Anybody else have a problem with truncated log files using SAS UE batch submit?

Hi, @Damien_Mather! I had one of my colleagues who is very skilled in SAS try to replicate the problem and she wasn't able. There must be some other aspect that we need to touch upon.

 

Here are some additional questions for you to address at your leisure: Are you using VMware or VirtualBox?  Are you looking at the log from within SAS Studio?  Is there anything in there to indicate it’s truncated? Can you try looking at it outside of SAS Studio?

 

Thanks,

Randy

Frequent Contributor
Posts: 130

Re: Anybody else have a problem with truncated log files using SAS UE batch submit?

Hi Randy.

 

Actually according to my doctor I've been fighting off 3 'fresher' colds in as many weeks - it's spring here in the southern hemisphere - the down side of lots of student contact?

 

I strongly suspect the log silencing is something to do with the way that the SAS-supplied  %choiceff macro handles log and output - especially ods output to log destination - that interacts with the studio batch submit mode.

 

The last block of code that is sumbitted from my program before the log goes quiet is:

 


%macro itchoeff(qf=,q2=, qby=, mynalts=);
%do p2 = &qf. %to &q2. %by &qby.;
%choiceff
(
data=perm.yogurt,
model=
class(brand3 feature / effects)
identity(price)
,
nsets=&p2,
nalts=&mynalts.,
seed=09051958,
options=nodups,
maxiter=1,
beta=zero
);
%end;
%mend;
%itchoeff( qf=2475,q2=2475,qby=1,mynalts=&nupc.);

 

 

which is me wapping a grid search for a balanced brand choice design around the %choiceff macro to search a big scan panel data set for one or more balanced choice designs.

 

I seem to recall these macros interacting with sas logs in other batch submit scenarios, but that was too long ago to be sure. 

 

It only happens in batch - in interactive mode all log and output is generated normally.

Super User
Super User
Posts: 6,500

Re: Anybody else have a problem with truncated log files using SAS UE batch submit?

That macro is calling:

proc printto log=_dummy; run;

So if it crashed in an unexpected way then your log is still being written to dummy.

Add this line after you call it and at least then you wll see the log from your statements.

proc printto log=log; run;

It can also generate:

OPTIONS NONOTES;

So you should also add this line after you call it, just in case it bombs and does not get to reset the option back.

OPTIONS NOTES;
Frequent Contributor
Posts: 130

Re: Anybody else have a problem with truncated log files using SAS UE batch submit?

Hi Tom.

 

Thanks for pointing that out. That would explain it. 

 

In interactive mode there is no indication of any error or abnormal end to the macro execution, but that doesn't mean that does not happen, right? 8-)

 

I'll apply your suggestions and let you know how they go.

 

thanks again

 

Damien

Frequent Contributor
Posts: 130

Re: Anybody else have a problem with truncated log files using SAS UE batch submit?

Nice try Tom but the truncation persists! I still believe that it is the log destination redirection in the %choiceff macro that is causing the problem.

 

The last steps in my code that are reflected in the log are now:

.

 


%macro itchoeff(qf=,q2=, qby=, mynalts=);
%do p2 = &qf. %to &q2. %by &qby.;
%choiceff
 (
  data=perm.yogurt,
  model=
   class(brand3 feature / effects)
   identity(price)
  , 
  nsets=&p2,
  nalts=&mynalts.,
  seed=09051958,
  options=nodups,
  maxiter=1,
  beta=zero
 ); 
 proc printto log=log;
 run;
 options notes;
 run; 
%end;
%mend;
%itchoeff( qf=1660,q2=2172,qby=256,mynalts=&nupc.);

But building on your suggestion, I think I'll try substituting the fully specified destination of the log file as initially generated by the studio batch submit:

 

proc printto log='/folders/myfolders/research/yog/programs/yog3.log';
run;

 

Fingers crossed!

 

Frequent Contributor
Posts: 130

Re: Anybody else have a problem with truncated log files using SAS UE batch submit?

The workaround I adopted was to extend the SAS UE interactive session timeout setting from the default 30 minutes to a much larger time. I'm embarrassed to say that some of my proc glimmix models have taken days to converge and terminate normally, and it was nice to see that the SAS UE settin for this can be extended to some big number of hours

Regular Contributor
Posts: 231

Re: Anybody else have a problem with truncated log files using SAS UE batch submit?

Where is this setting?  That would be good for some of the students I've talked with to know about.  

thanks for posting back!

Chris

Has my article or post helped? Please mark as Solution or Like the article!
Solution
‎11-25-2016 09:37 PM
Frequent Contributor
Posts: 130

Re: Anybody else have a problem with truncated log files using SAS UE batch submit?

I found it under the 'more application options' tool button immediately to the left of 'help' tool button, in the left hand side of the studio tool bar. The tool button is the one that looks like 3 horizontally stacked books with an elipsis ... underneath. Select Preferences... . Under the General tab there is a Time-out interval setting which you can increment or decrement in 0.1 hour units using up or down arrow buttons.

 

The locked-down states of SAS UE studio and SAS OnDemand Studio differ in how big you can make this timeout, which makes sense. My CPU time is my own, whereas SAS OD's CPU time is generously donated to students by SAS! SAS OD Studio timeout setting stops at a maximum of 3.0 hours, whereas SAS UE Studio timeout goes way beyond that. I will leave it as an exercise for the reader to see just how stupidly large one can set it in SAS UE.

 

Regular Contributor
Posts: 231

Re: Anybody else have a problem with truncated log files using SAS UE batch submit?

@Damien_Mather very cool and thanks for the info.  will definitely be playing with it as i frequently get error messages that the data is too big or too complex - hoping that this tweak resolves that!

 

have a good afternoon / evening Smiley Happy

Chris

Has my article or post helped? Please mark as Solution or Like the article!
☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 10 replies
  • 698 views
  • 2 likes
  • 4 in conversation