topic Re: issue with dosubl, proc document and titles in SAS Programming

issue with dosubl, proc document and titles

ecofred — Fri, 06 Sep 2019 10:27:05 GMT

Given the 2 files

--- doc_test.sas ----

ods document name=testdoc(write);

	title1 j=c 'CARS' j=r 'page 0 of 100';
	title2 'this is title 2';
	footnote 'this is a footnote';
	
	proc contents data=sashelp.cars; run;
	
ods document close;

Title 'WHERE IS THE OBTITLE???';
proc document name=testdoc(read);
	list / details levels=all;
	obtitle \Contents#1\DataSet#1\EngineHost#1 show;
	replay;
run;
quit;

-- procdoc_subl.sas --

Title; Footnote;
%* find the parent folder ------;
data _null_;
	dir = prxchange('s/[\w\.]*$//',-1, "&_SASPROGRAMFILE.");
	call symputx('mydir', dir);
run;


data _null_;
	rc=dosubl("%include '&mydir.doc_test.sas' / source2;");
run;

if I run 'procdoc_subl.sas' (with 'doc_test.sas' in the same folder) in SAS Studio in the SAS University Edition,

then the dosubl part fail to 'SHOW' / or find the 'OBTITLE' are any kind of title from the ItemStore.

Also the replay is missing the Titles

FCMP/RUN_MACRO or RUN_SASFILE also fails

Can anyone reproduce this issue? Or have any idea what is missing?

Re: issue with dosubl, proc document and titles

JosvanderVelden — Fri, 06 Sep 2019 15:07:16 GMT

Can you verify the code in procdoc_subl.sas?
I think it is not correct.

Re: issue with dosubl, proc document and titles

ecofred — Fri, 06 Sep 2019 19:10:43 GMT

I did check again; the program works if the 2 are in the same folder and run in SAS Studio.
Can point to the incorrect element?
I will add the log for 'procdoc_subl.sas' in the thread.

Re: issue with dosubl, proc document and titles

ecofred — Fri, 06 Sep 2019 19:14:28 GMT

-------- the log (with option added to show macro elements) ---------------

 1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 70         
 71         option MLOGIC SYMBOLGEN MPRINT;
 72         Title; Footnote;%* find the parent folder ------;
 73         data _null_;
 74         dir = prxchange('s/[\w\.]*$//',-1, "&_SASPROGRAMFILE.");
 SYMBOLGEN:  Macro variable _SASPROGRAMFILE resolves to /home/fclems0/procdoc_dosubl.sas
 SYMBOLGEN:  Some characters in the above value which were subject to macro quoting have been unquoted for printing.
 75         call symputx('mydir', dir);
 76         run;
 
 NOTE: DATA statement used (Total process time):
       real time           0.00 seconds
       user cpu time       0.00 seconds
       system cpu time     0.00 seconds
       memory              490.93k
       OS Memory           29348.00k
       Timestamp           09/06/2019 07:01:54 PM
       Step Count                        45  Switch Count  0
       Page Faults                       0
       Page Reclaims                     97
       Page Swaps                        0
       Voluntary Context Switches        0
       Involuntary Context Switches      0
       Block Input Operations            0
       Block Output Operations           0
       
 
 77         
 78         data _null_;
 79         rc=dosubl("%include '&mydir.doc_test.sas' / source2;");
 SYMBOLGEN:  Macro variable MYDIR resolves to /home/fclems0/
 80         run;
 
 NOTE: %INCLUDE (level 1) file /home/fclems0/doc_test.sas is file /home/fclems0/doc_test.sas.
 NOTE: PROCEDURE CONTENTS used (Total process time):
       real time           0.07 seconds
       user cpu time       0.07 seconds
       system cpu time     0.00 seconds
       memory              3794.96k
       OS Memory           31400.00k
       Timestamp           09/06/2019 07:01:54 PM
       Step Count                        46  Switch Count  0
       Page Faults                       0
       Page Reclaims                     363
       Page Swaps                        0
       Voluntary Context Switches        5
       Involuntary Context Switches      0
       Block Input Operations            0
       Block Output Operations           136
       
 
 NOTE: PROCEDURE DOCUMENT used (Total process time):
       real time           0.09 seconds
       user cpu time       0.09 seconds
       system cpu time     0.00 seconds
       memory              3805.59k
       OS Memory           31652.00k
       Timestamp           09/06/2019 07:01:54 PM
       Step Count                        46  Switch Count  0
       Page Faults                       0
       Page Reclaims                     71
       Page Swaps                        0
       Voluntary Context Switches        0
       Involuntary Context Switches      0
       Block Input Operations            0
       Block Output Operations           240
       
 
 NOTE: %INCLUDE (level 1) ending.
 NOTE: DATA statement used (Total process time):
       real time           0.17 seconds
       user cpu time       0.17 seconds
       system cpu time     0.00 seconds
       memory              3805.59k
       OS Memory           31652.00k
       Timestamp           09/06/2019 07:01:54 PM
       Step Count                        46  Switch Count  12
       Page Faults                       0
       Page Reclaims                     622
       Page Swaps                        0
       Voluntary Context Switches        92
       Involuntary Context Switches      0
       Block Input Operations            0
       Block Output Operations           496
       
 
 81         
 82         OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 SYMBOLGEN:  Macro variable GRAPHTERM resolves to GOPTIONS NOACCESSIBLE;
 93

Re: issue with dosubl, proc document and titles

JosvanderVelden — Wed, 11 Sep 2019 09:51:15 GMT

My remark with the code has to do with the fact that I don't understand where you declare the _SASPROGRAMFILE macro variable.

I also suspect the title1 statement is an issue.

I suggest to change the code for doc_test.sas to:

ods document name=testdoc(write);

	title1 j=c 'CARS';
	title2 'this is title 2';
	title3 j=r 'page 0 of 100';
	footnote 'this is a footnote';
	
	proc contents data=sashelp.cars; run;
	
ods document close;

Title 'WHERE IS THE OBTITLE???';
proc document name=testdoc(read);
	list / details levels=all;
	obtitle \Contents#1\DataSet#1\EngineHost#1 show;
	replay;
run;
quit;

Maybe that gives you a clue for further analysis.

Best regards, Jos

Re: issue with dosubl, proc document and titles

ecofred — Wed, 11 Sep 2019 13:50:39 GMT

Dear Jos,

Multiple J= seems to be a SAS feature (http://support.sas.com/kb/24/492.html) and the effect I wanted: both on the same line (see picture below to illustrate that). I don't want to loose 1 line on my page 🙂

I just noticed that the standard HTML/'Result' get it a bit funky with multiple J=: they maybe way over on the right.

But I would not relate it to my question: why does proc document forget about OBTITLE with DOSUBL/%include ?

However I did give a try to no multi j= in title1, but the obtitle are still missing when I want to use/list them with the dosubl

The &_SASPROGRAMFILE. macro is set up by the SAS Studio Ondemand University edition session (I used https://odamid.oda.sas.com/SASStudio to make it reproducible). It is referenced in http://support.sas.com/kb/24/301.html.(this is one of the reason why I specified working in SAS Studio). But I could indeed have commented that in my code.

-- procdoc_subl.sas -- v2 with comments and option to expand macro in log.

option MLOGIC SYMBOLGEN MPRINT;

%*
this program make use of the SAS Studio '_SASPROGRAMFILE' macro, thus
run only in SAS Studio ( SAS® Enterprise Guide seems also ok)
;

Title; Footnote;

%* find the parent folder ------;
data _null_;
	dir = prxchange('s/[\w\.]*$//',-1, "&_SASPROGRAMFILE.");
	call symputx('mydir', dir);
run;

data _null_;
	rc=dosubl("%include '&mydir.doc_test.sas' / source2;");
run;

Best Regards,

frederic.

Re: issue with dosubl, proc document and titles

JosvanderVelden — Wed, 11 Sep 2019 15:41:31 GMT

What happens when you change the doc_test.sas in your environment to:

ods document name=testdoc(write);

	title1 j=l 'CARS' j=c 'page 0 of 100';
	title2 j=c 'CARS' j=r 'page 0 of 100';
	title3 j=l 'CARS' j=r 'page 0 of 100';
	footnote 'this is a footnote';
	
	proc contents data=sashelp.cars; run;
	
ods document close;

Title4 'WHERE IS THE OBTITLE???';
proc document name=testdoc(read);
	list / details levels=all;
*	obtitle \Contents#1\DataSet#1\EngineHost#1 show;
	replay / actitle acfootn;
run;
quit;

Re: issue with dosubl, proc document and titles

Quentin — Wed, 11 Sep 2019 15:48:37 GMT

I don't know much about PROC DOCUMENT, but I've been trying to figure out various corners of DOSUBL, so this is interesting. I'm running 9.4M4 on windows, display manager SAS. I tried a simplified test, and can't explain the result.

Following code creates a document:

ods document name=testdoc(write);
  title1 "My custom title for CARS";
  footnote "My custom footnote for CARS";
  proc print data=sashelp.cars(obs=3); 
  run;
  title1 ;
  footnote1 ;
ods document close;

I assume the titles and footnotes are stored as part of the document, right?

If I then replay the document, I get the titles and footnotes:

proc document name=testdoc(read);
	replay;
run;
quit;

If use replay the document in the DOSUBL side session, I don't get the titles and footnotes, surprisingly:

data _null_;
	rc=dosubl('proc document name=testdoc(read);replay;run;');
run;

Clearly the document is read, just somehow the titles/footnotes aren't being read, or are being read but aren't being honored (?).

With your %include approach, you create the document and replay it in the DOSUBL side session. I tried that too, without the %include. Same result (no titles/footnotes) on the replay:

data _null_;
  rc=dosubl('
ods document name=testdoc(write);
  title1 "My custom title for CARS";
  footnote "My custom footnote for CARS";
  proc print data=sashelp.cars(obs=3); 
  run;
  title1 ;
  footnote1 ;
ods document close;

proc document name=testdoc(read);
	replay;
run;
quit;  
');
run;

I would have hoped all three of those code blocks would show the titles when replayed. I would suggest sending this in to tech support, to ask why PROC DOCUMENT isn't reading the titles when it is called inside DOSUBL.

There is a lot of magic in DOSUBL, but also some interesting side effects.

Re: issue with dosubl, proc document and titles

Tom — Wed, 11 Sep 2019 16:49:41 GMT

Looks like the issue is being caused by use of DOSUBL() function.

Are you just curious about why DOSUBL() is causing this issue? Or do you have an actual need to get your program to run?

If the later then what is the reason that your program is trying to use DOSUBL() instead of just %INCLUDE, CALL EXECUTE(), macro code or other code generation methods?

Re: issue with dosubl, proc document and titles

Quentin — Wed, 11 Sep 2019 17:19:39 GMT

Well, I'm curious every time I'm surprised by DOSUBL behavior. The docs are fairly light as to what it's fair to expect from DOSUBL . = )

Re: issue with dosubl, proc document and titles

Quentin — Wed, 11 Sep 2019 18:42:17 GMT

If I code an explicit ODS statement, I do get the title in the PDF.

Even when the ODS statement is outside of the DOSUBL block.

ods pdf file="%sysfunc(pathname(work))\foo.pdf" ;
data _null_;
  rc=dosubl('
    proc document name=testdoc(read);
      replay;
    run;
    quit ;
  ');
run;
ods pdf close ;

So looks like PROC DOCUMENT may be doing it's part, but DOSUBL is having a hard time returning titles to the default output (?). In my case, my default output destination is Display Manager .lst, assume in Studio you're getting some flavor of ODS HTML.

Re: issue with dosubl, proc document and titles

ecofred — Wed, 11 Sep 2019 20:16:26 GMT

dear Jos,

I'm aware of the active*, I tried it before starting this post and it is not reliable, It messes with the font/style.

But my main issue with the active* in the current context, is that I then have to rely on the right title and footnotes being loaded at the right time and the right place once again.

It is an asumption I can not rely on because between the ods document and the replay, some titles may have changed, and it misses the point of an ods item being independant from the rest. All Batterie included is what I like about the item store.

This is the reason why I have

Title 'WHERE IS THE OBTITLE???';

in the middle of 'doc_test.sas', To also check if the Titles from the ods object are reused instead of the current session title.

Also, setting element title, where page break is 'before', with the obtitle statement is not a viable solution:

- it is not multi j= compatible (as far as I know)

- requires to set again what is expected to have already been done.

My issue/setup is quite convoluted... but I would still expect it to work.

It could technically be refactored, but this is not an option :(. I'm locked with it for some external reasons, but that is an other story.

Best Regards,

Frederic.

Re: issue with dosubl, proc document and titles

ecofred — Wed, 11 Sep 2019 21:57:14 GMT

Dear Tom,

Yes I do have a practical need, but i'm also curious because it does not fit in my current model of how DOSUBL (and RUN_FILE) works.

I have a list of output names in a dataset, each output is associated to a file. Each files produce 3 things: a dataset, a ods document object, a pdf (replay).

With a DATA STEP, I visit the dataset, infer the path to each output files, give it to dosubl/%include, check that the files where produce and create new variable to report the success/failure for the production of each outputs.

An other step is to let run a QC program. this gives 'only' a dataset for each output. I can then dosubl or run_macro to capture the &SYSINFO. value of the proc compare.

Compared to the call execute, I get better encapsulation/local properties for my processes and can streamline the success/failure directly to the PDV (did I mentionned how gorgious i find the DOSUBL/RUN_* function?). all in one path.

With call execute I have to insert the progresses to another dataset, open it many times and take the risk of having some concurrency issues.

From experience I find DOSUBL more reliable than CALL EXECUTE,

Title 'DOSUBL = reliable';
data zz;
	rc=dosubl("proc contents data=sashelp.cars short; run';");
	output;
	rc=dosubl("proc contents data=sashelp.airline short; run;");
	output;
run;

Title 'CALL EXECUTE leak: AIRLINE missing!';
data zz;
	call execute("proc contents data=sashelp.cars short; run';");
	call execute("proc contents data=sashelp.airline short; run;");
run;

Please let this code illustrate my point:. a typo is placed in the form of *run';* in the 2 datasteps. luckily for CALL EXECUTE, the 2nd instruction is not mission critical... (did I mentionned that I can not trust the output programs).

So technically, this dosubl/proc document issue is the last challenge in an attempt of abstracting the full processes line (output generation + Quality check) in what I would define as a reliable and metadata-driven and BASE SAS only solution.

Now, in a sane world, I would refactor more than one part and make it work, but I'm not living in a sane world, and I can not touch programs generating the outputs.(lack of time and QC issue are the 2 main reasons).

I currently have a call execute as a 2d pass to correct the outputs, (following the 'first make it work' principle 🙂 ). But it seems so close,..

Regards,

Frederic

Re: issue with dosubl, proc document and titles

ecofred — Wed, 11 Sep 2019 22:17:34 GMT

Dear Quentin, I answered to Tom below, it may give some context.

Yes, it seems that our assumption are not up to the fact 😞

I ran your 3 versions one after an other in a same .sas (I reset title and footnote after each) in SAS Studio

In my SAS Studio session, the 1 and 2 succeed, but... of course ... the 3rd fails... (see the last page of the pdf)

Kind regards,

fred

Re: issue with dosubl, proc document and titles

Quentin — Wed, 11 Sep 2019 23:49:38 GMT

Interesting, I'm surprised #2 succeeded.

I suggest sending something simple like #3 to tech support. I think we should expect that code executed by DOSUBL() will produce the same results as code executed outside of DOSUBL(), until/unless there is documentation to explain the difference (or acknowledgement that it's a bug).

Re: issue with dosubl, proc document and titles

ecofred — Thu, 12 Sep 2019 05:50:04 GMT

The doc is fine, but the notes/issues on dosubl shows that this is not the only limitation. The closest one to my issue may well be http://support.sas.com/kb/55/931.html .

But I should give support a try. I exhausted all the other ressources. I have to give up expected reliability for a working solution... Muhh :(... But the hope should stay

Re: issue with dosubl, proc document and titles

Quentin — Fri, 27 Sep 2019 19:50:16 GMT

@ecofred Did you hear anything interesting from tech support?