topic proc surveyreg - speed in Statistical Procedures

proc surveyreg - speed

Sinistrum — Wed, 28 Feb 2018 16:36:55 GMT

Dear community,

I would like to ask for your help concerning the following issue.

For the case at hand, I need to run regressions with fixed effects and clustered standard errors. Up to now, I am mainly interested in the estimated parameters.

Thus, I ran something like

proc	surveyreg
	data				=			in;
	class 							classvar;
	cluster 						clustervar;
	model
		y				= 			x1	x2			 	
									classvar	
									/ solution;
	ods output
	ParameterEstimates 	= 			param;
run;

I use this highly esteemed tool "to turn ODS off".

Though, I do not seem to find a "noprint" option for "proc surveyreg" with an corresponding "outX=" for the parameter estimates.

I would be glad, if you could help with if there are any means by which I can possible increase the speed by which the regression, i.e., a regression with fixed effects and clustered standard errors, is performed?

Are there other procedures or may I modify the statement above, sticking to "proc surveyreg"?

Yours sincerely,

Sinistrum

Re: proc surveyreg - speed

Rick_SAS — Thu, 01 Mar 2018 18:11:25 GMT

How many levels are in the CLASSVAR variable?

As for alternatives, you might want to read this related thread. The situation is slightly different than your, but some of the ideas might be relevant.

Re: proc surveyreg - speed

ballardw — Thu, 01 Mar 2018 18:33:16 GMT

Some of your "speed" issue might be related to tables that you aren't interested in when generating the output. It seems that a number of these procedures build everything in memory and then send them to the results window.

You might get an improvement by using either ODS SELECT or ODS EXCLUDE to restrict how much output is sent to the results.

These would be similar to the "noprint" of older procedures. ODS Select would set which tables to keep in your output or ODS Exclude would suppress a list.

If you don't know the names of tables to select/exclude you could either run the procedure with a reduced data set and the ODS Trace option:

ods trace on;

proc surveyreg data=yourdata (where = (classvar=1));

ods trace off;

which will generate a log entry with the tables used.

Re: proc surveyreg - speed

Sinistrum — Fri, 02 Mar 2018 07:20:24 GMT

First, thank you for your answers.

Rick_SAS wrote:
How many levels are in the CLASSVAR variable?

Around 1200. I run panel regressions.

Thank you for the link to the other thread.

ballardw wrote:
You might get an improvement by using either ODS SELECT or ODS EXCLUDE to restrict how much output is sent to the results.

I modified the code to

proc	surveyreg
	data				=			in;
	class 							classvar;
	cluster 						clustervar;
	model
		y				= 			x1	x2			 	
									classvar	
									/ solution;
	ods	select
	ParameterEstimates;
	ods output
	ParameterEstimates 	= 			param;
run;

If by means of this I implemented your suggestion correctly, I were not capable of reducing the speed it took to perform the calculations.

I assume the problem seems to be (which goes into the direction of the issue discussed in the thread Rick_SAS linked to; confer the last post there) that proc surveyreg performs a "Least Square Dummy Variable" approach and thus calculates a coefficient for every individual of the classvar. This would suggest it is not a problem of creating redundant output though indeed a matter of parameter estimation.