proc ttest NO OUTPUT CREATED

[Gigi2021]

proc datasets library=WORK kill; run; quit;

libname Cooked "d:\Data\";

/* Import Trade data and compute some variables */
data TrdGrpDataTemp;
set cooked.DrkTradesPlus_TrdGrp1;
sprd = (log(ask) - log(bid))*100;
sprd5 = (log(askpi5) - log(bidpi5))*100;
sprd30 = (log(askpi30) - log(bidpi30))*100;
PI5 = abs(log(askpi5 + bidpi5)-log(ask+bid))*100;
PI30 = abs(log(askpi30 + bidpi30)-log(ask+bid))*100;
PR = abs(log(askpi5 + bidpi5)-log(ask+bid)-(log(askpi30 + bidpi30)-log(askpi5+bidpi5)))*100;
if sign(log(askpi5 + bidpi5)-log(ask+bid)-1) = sign(log(askpi30 + bidpi30)-log(askpi5+bidpi5)-1) then PRI = 1; else PRI = 0;
PAS = divide((askpi30 + bidpi30)/2 - (askpi5 + bidpi5)/2,(ask + bid)/2)*100;
if ask < bid then delete;/* Filter1a */
if askpi5 < bidpi5 then delete;/* Filter1b */
if askpi30 < bidpi30 then delete;/* Filter1c */
if date > "30SEP2019"d then delete; /* Delete the last couple of days. */
if venue = "UBSP" OR venue = "CGMA" OR venue = "2011" OR venue = "2311" OR venue = "3611" OR venue = "MACB" then ifnoHFT = 1;
else if not missing(venue) then ifnoHFT = 0;
if missing(venue) then delete;
run;

/* t-test */
ods exclude all;
proc ttest data = TrdGrpDataTemp;
var val sprd price sprd5 sprd30 pi5 pi30 pr pas;
class ifnoHFT;
ods output ttests = Summary_test equality  = summary_test2 statistics = Summary_test1;
run;
ods exclude none;

Hi, All, 

The above code went through; but when I replaced the second line by a larger sized dataset (of cos, exactly the same data structure. In fact, the larger one contains all observations in the small dataset), i.e., change "cooked.DrkTradesPlus_TrdGrp1" into "cooked.DrkTradesPlus_AllGrp". I have got the following error messages. 

WARNING: Output 'statistics' was not created.  Make sure that the output object name, label, or
         path is spelled correctly.  Also, verify that the appropriate procedure options are used
         to produce the requested output object.  For example, verify that the NOPRINT option is
         not used.
WARNING: Output 'equality' was not created.  Make sure that the output object name, label, or
         path is spelled correctly.  Also, verify that the appropriate procedure options are used
         to produce the requested output object.  For example, verify that the NOPRINT option is
         not used.
WARNING: Output 'ttests' was not created.  Make sure that the output object name, label, or path
         is spelled correctly.  Also, verify that the appropriate procedure options are used to
         produce the requested output object.  For example, verify that the NOPRINT option is not
         used.

I have googled and read some of posts in this forum, but I could not figure out why. Here is the list I troubleshoot:

1. Some posts say it is possible because the level of "class" statement is not 2 any more. 

- It is not applied in my case. Because the larger dataset contains all observations in the "TrdGrp1" dataset;

2. Some posts say sth to do with ODS output. 

- From my understanding, I have correctly specified to make sure the place to ODS is correct. 

Any suggestions, please?

[PeterClemmensen]

If the larger contains all observations in the smaller data set and more, it seems that (1) is indeed possible?

[Gigi2021]

Why?

The small dataset went through and it means that the "class" does have two levels. 

[andreas_lds]

I am not familiar with proc ttest, but i would not expect any output from the procedure, due to ods exclude all.

[FreelanceReinh]

Hi @Gigi2021,

It's always best to show the full log of a step that doesn't work as expected, ideally including the code as it appears in the log. The warning messages you've shown don't tell the full story. For example, they might be preceded by

ERROR: The CLASS variable does not have two levels.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE TTEST used (Total process time):
...

or

ERROR: The CLASS variable has more than two levels.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE TTEST used (Total process time):
...

Obviously, these error messages would be very helpful to understand what has happened. In particular, they would immediately decide the question whether the CLASS variable has exactly two levels.

[Gigi2021]

That’s the full log. No more message.

[FreelanceReinh]

---

@Gigi2021 wrote:
That’s the full log. No more message.

---

This indicates that parts of the messages were suppressed by system options. Therefore please submit

options notes source;

rerun the PROC TTEST step and the surrounding ODS EXCLUDE statements and post the full log of that including the code.

[Gigi2021]

If I use the small dataset, after running PROC TTEST, I will get the following:

NOTE: The data set WORK.SUMMARY_TEST1 has 27 observations and 14 variables.
NOTE: The data set WORK.SUMMARY_TEST2 has 9 observations and 6 variables.
NOTE: The data set WORK.SUMMARY_TEST has 18 observations and 6 variables.
NOTE: PROCEDURE TTEST used (Total process time):
      real time           0.36 seconds
      cpu time            0.04 seconds

If I use the large dataset, I'll see the followiing:

WARNING: Output 'statistics' was not created.  Make sure that the output object name, label, or
         path is spelled correctly.  Also, verify that the appropriate procedure options are used
         to produce the requested output object.  For example, verify that the NOPRINT option is
         not used.
WARNING: Output 'equality' was not created.  Make sure that the output object name, label, or
         path is spelled correctly.  Also, verify that the appropriate procedure options are used
         to produce the requested output object.  For example, verify that the NOPRINT option is
         not used.
WARNING: Output 'ttests' was not created.  Make sure that the output object name, label, or path
         is spelled correctly.  Also, verify that the appropriate procedure options are used to
         produce the requested output object.  For example, verify that the NOPRINT option is not
         used.
NOTE: PROCEDURE TTEST used (Total process time):
      real time           1:19.93
      cpu time            1:17.84

It looks to me PROC TTEST was exectued. In your opinion, what's the issue?

[FreelanceReinh]

The log doesn't show the submitted code, so I can't perform the checks suggested in the warning messages.

As the problem seems to be data dependent, let's take a look at the data. Please replace "ttest" by "means" in a copy of your code and remove all ODS statements:

proc means data = TrdGrpDataTemp;
var val sprd price sprd5 sprd30 pi5 pi30 pr pas;
class ifnoHFT;
run;

The output of this step, based on the "large" dataset, might provide useful insights.

[STAT_Kathleen]

Does it make any difference if you specify the CLASS statement prior to the VAR statement?  For example:

proc ttest data = TrdGrpDataTemp;

class ifnoHFT;
var val sprd price sprd5 sprd30 pi5 pi30 pr pas;
ods output ttests = Summary_test equality  = summary_test2 statistics = Summary_test1;
run;

If the error still occurs, please provide the complete log .

[Gigi2021]

@FreelanceReinh regarding two suggestions from you:

1. replace "proc ttest" by "proc means": so it went through if I use "proc means". No errors;

2. change the position of "class": it does not make difference. Still same error message.

[FreelanceReinh]

---

@Gigi2021 wrote:

@FreelanceReinh regarding two suggestions from you:

1. replace "proc ttest" by "proc means": so it went through if I use "proc means". No errors;

2. change the position of "class": it does not make difference. Still same error message.

---

Great to hear back from you.

The interesting result of the PROC MEANS step would be its output. (Can you show it, with confidential information removed or blurred, if necessary?) I'm not surprised that the log (of such a simple step) is clean and also that the position of the CLASS statement doesn't make a difference (which wasn't my suggestion anyway).

[Gigi2021]

@FreelanceReinh , log window shows the following:

418  proc means data = TrdGrpDataTemp;
419  var val sprd price sprd5 sprd30 pi5 pi30 pr pas;
420  class ifnoHFT;
421  run;

NOTE: There were 42990468 observations read from the data set WORK.TRDGRPDATATEMP.
NOTE: PROCEDURE MEANS used (Total process time):
      real time           44.52 seconds
      cpu time            52.81 seconds

Output window is blank.

Results window shows the following:

[FreelanceReinh]

Thank you very much. So, apparently it should be possible in principle to perform the t-tests on your data, but PROC TTEST (with the ODS OUTPUT statement) doesn't do it if the input dataset is "too large."

Indeed, I've just created a test dataset with 25 million observations and my SAS 9.4M5 fails to produce output as well and writes the three warnings to the log.

But @STAT_Kathleen's suggestion to turn ODS GRAPHICS off resolves the issue.

Edit: ODS GRAPHICS can also have a disastrous impact on run time (see https://communities.sas.com/t5/SAS-Programming/how-slow-would-the-TTest-procedure-be/m-p/488900/high... from 2018 for an example with PROC TTEST), but I was under the impression that your ODS EXCLUDE statement would counteract that effect sufficiently. Good to know that this is not the case. Rick_SAS's macros ODSOff and ODSOn mentioned in that 2018 thread also use both these ODS statements among others in order to improve performance.

Edit 2: As described in later posts of that 2018 thread, with older releases such as SAS 9.4M2 it's even possible that SAS works unnecessarily on certain ODS-related files and thus wastes time and disk space in spite of ods graphics off.

[Gigi2021]

Hi, everyone who ever viewed my question a couple of days ago, here are some updates:

1. I suspected the problem has something to do with the limit SAS can handle (i mean, beyond a certain operational capacity?). This is because, I have tried to load the full dataset which has 43,456,167 observations. Then, if I put a line "if _n_ < 20000000" in the DATA step before running "PROC TTEST", the code gave me output. But, if I change the threshold higher,  "if _n_< 25000000" does not give me output;

2. Based on the log which says "PROCEDURE TTEST used", (together with @FreelanceReinh 's suggestion on replacing PROC TTEST by PROC MEANS to troubleshoot, I have a feeling the problem is also associated with the procedure TTEST itself and its interaction with ODS. 

Anyone is aware the data processing limit? or any suggestions?