topic Re: proc reg error in SAS Programming

proc reg error

ninemileshigh — Tue, 10 Oct 2017 07:47:13 GMT

I am running a programme which includes several thousand regressions and I need the ods outputs.. The programme has failed twice with the following log

ERROR: Freeing Memory Not Allocated Pool [00000000024F0210] Value [000000002C8D8070] WM Task
Pool:REG .

Traceback for Thread [0000000002598650] WM Task Pool:REG

0000000001FC878E 0000000023E64800 tkmk:tkBoot+0x16A4E
000000000377A101 0000000023E64808
sashost:Main+0x325E1
0000000006EE4C34 0000000023E64848 sasods:tkvercn1+0x73BF4
0000000007083BEA
0000000023E688B0 sasods:tkvercn1+0x212BAA
0000000006ED4B72 0000000023E6AB40
sasods:tkvercn1+0x63B32
000000002509D33A 0000000023E6AB48
sasreg:tkvercn1+0x11C2FA
0000000024FF865C 0000000023E6F330
sasreg:tkvercn1+0x7761C
0000000024F92CE1 0000000023E6F620
sasreg:tkvercn1+0x11CA1
0000000024F90C67 0000000023E6F6E0
sasreg:tkvercn1+0xFC27
0000000024F81E02 0000000023E6F770 sasreg:tkvercn1+0xDC2
00000000051FC5E7
0000000023E6F778 sasxshel:tkvercn1+0x4B5A7
000000002509D0E2 0000000023E6F890
sasreg:tkvercn1+0x11C0A2
0000000024F816F9 0000000023E6FBF0
sasreg:tkvercn1+0x6B9
00000000037589DB 0000000023E6FF20 sashost:Main+0x10EBB
000000000375E62D
0000000023E6FF50 sashost:Main+0x16B0D
0000000076B959CD 0000000023E6FF58
kernel32:BaseThreadInitThunk+0xD
0000000076DCA561 0000000023E6FF88 ntdll:RtlUserThreadStart+0x21

ERROR: An exception has been encountered.
Please contact technical support and provide them with the following traceback information:

The SAS task name is [REG]
ERROR: Write Access Violation REG
Exception occurred at (01FCA0AA)
Task Traceback
Address Frame (DBGHELP API Version 4.0 rev 5)
0000000001FCA0AA 0000000023E68820 tkmk:tkBoot+0x1836A
0000000001FC8480 0000000023E68870 tkmk:tkBoot+0x16740
000000000377A101 0000000023E68878 sashost:Main+0x325E1
0000000006ED3B44 0000000023E688B8 sasods:tkvercn1+0x62B04
000000002509D33A 0000000023E6AB48 sasreg:tkvercn1+0x11C2FA
0000000024FF7295 0000000023E6F330 sasreg:tkvercn1+0x76255
0000000024F92CE1 0000000023E6F620 sasreg:tkvercn1+0x11CA1
0000000024F90C67 0000000023E6F6E0 sasreg:tkvercn1+0xFC27
0000000024F81E02 0000000023E6F770 sasreg:tkvercn1+0xDC2
00000000051FC5E7 0000000023E6F778 sasxshel:tkvercn1+0x4B5A7
000000002509D0E2 0000000023E6F890 sasreg:tkvercn1+0x11C0A2
0000000024F816F9 0000000023E6FBF0 sasreg:tkvercn1+0x6B9
00000000037589DB 0000000023E6FF20 sashost:Main+0x10EBB
000000000375E62D 0000000023E6FF50 sashost:Main+0x16B0D
0000000076B959CD 0000000023E6FF58 kernel32:BaseThreadInitThunk+0xD
0000000076DCA561 0000000023E6FF88 ntdll:RtlUserThreadStart+0x21

NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set WORK.LINFLUX_FIT20 may be incomplete. When this step was stopped there
were 7026 observations and 10 variables.
WARNING: The data set WORK.LINFLUX_PARAM20 may be incomplete. When this step was stopped there
were 4684 observations and 10 variables.
WARNING: The data set WORK.MODEL_CO220 may be incomplete. When this step was stopped there were
2342 observations and 9 variables.
NOTE: PROCEDURE REG used (Total process time):
real time 1:40:32.08
cpu time 42:28.21

how do I fix this??

Re: proc reg error

ChrisBrooks — Tue, 10 Oct 2017 08:07:58 GMT

It looks to me like some sort of memory overflow - could you reduce the amount of data you're passing through? If that doesn't fix it or you can't do it then I think you'll have to raise a track with SAS Support which you can do here http://support.sas.com/ctx/supportform/createForm

Re: proc reg error

Ksharp — Tue, 10 Oct 2017 11:33:34 GMT

Try to use BY statement of PROC REG .

Re: proc reg error

ninemileshigh — Tue, 10 Oct 2017 11:42:21 GMT

thanks. I've put an extra loop into my macro so that it executes the proc reg in smaller chunks of data, to see if that will work

Re: proc reg error

Ksharp — Tue, 10 Oct 2017 11:47:53 GMT

https://blogs.sas.com/content/iml/2017/02/13/run-1000-regressions.html

Re: proc reg error

Rick_SAS — Tue, 10 Oct 2017 13:18:10 GMT

You should "contact technical support and provide them with the following traceback information" in addition to your SAS version, operating system, and program.

AS KSharp said, you probably want to use BY-group processing for this problem, rather than a macro loop. And make sure that you use ODS GRAPHICS OFF. Also use NOPRINT or ODS EXCLUDE ALL to suppress the SAS output.

Re: proc reg error

ninemileshigh — Tue, 10 Oct 2017 13:23:25 GMT

thanks for the link, it will come in useful for future. I think I've worked around the problem by analysing smaller chunks of data for now.

Re: proc reg error

Rick_SAS — Tue, 10 Oct 2017 13:25:16 GMT

Please post your SAS program. We might be able to offer a more efficient solution.

Re: proc reg error

ninemileshigh — Tue, 10 Oct 2017 13:26:15 GMT

thanks, I'll send that through. I have managed a temporary fix by analysing smaller chunks of data.. (a lazy way out, I know). I can't use noprint as I need the ods tables output.

Re: proc reg error

ninemileshigh — Tue, 10 Oct 2017 13:28:45 GMT

here's the code:

/****************************************************************
* NOW CODE A MACRO TO REGRESS CO2 FOR 60 SECOND LINEAR FIT WITH *
* A MOVING STARTING TIME EVERY 15 SECONDS *
****************************************************************/

/* DIVERT LOG TO A FILE FOR THE BATCH PROCESSING- ELSE IT HALTS THE PROGRAMME WHEN IT GETS FULL */

proc printto new log="&OUTFOLDER.\CRICHTONLINEARlog.LOG" print="&OUTFOLDER.\CRICHTONLINEAR.LOG";
run;

%MACRO WINDOWS;

%DO h= 01 %TO 21;

%DO i= 20 %TO 170 %BY 15;

DATA START&h._&i.;
SET DATASET.ONE;
IF ETIM < &i. THEN DELETE;
IF ETIM > &i. + 60 THEN DELETE;
IF LICORFILECODE NE &h. THEN DELETE;
RUN;

PROC REG DATA= START&h._&i. OUTEST= MODEL_CO2&h._&i. ;
MODEL CO2_uMOL= ETIM;
ODS OUTPUT PARAMETERESTIMATES= LINFLUX_PARAM&h._&i. FITSTATISTICS= LINFLUX_FIT&h._&i.;
BY LICORFILE OBS;
RUN;

DATA FLUX_CO2&h._&i.;
SET MODEL_CO2&h._&i.;
LINFLUX= ETIM/ 0.12;
DEADBAND= &i.;
PROC PRINT;
RUN;

DATA LINFLUX_FIT2_&h._&i.;
SET LINFLUX_FIT&h._&i. (RENAME=(cValue2= LINFLUX_R2));
IF Label1 ^= 'Dependent Mean' THEN DELETE;
DEADBAND= &i.;
RUN;

DATA LINFLUX_CV&h._&i.;
SET LINFLUX_FIT&h._&i. (RENAME=(cValue1= LINFLUX_CV));
IF MISSING(Label2)= 0 THEN DELETE;
DEADBAND= &i.;
RUN;

DATA LINFLUX_PARAMS&h._&i.;
SET LINFLUX_PARAM&h._&i. (RENAME=(tValue= LINFLUX_T Probt= LINFLUX_P));
IF Variable = 'Intercept' THEN DELETE;
DEADBAND= &i.;
RUN;

DATA LINEARFIT&h._&i.;
MERGE FLUX_CO2&h._&i. LINFLUX_FIT2_&h._&i. LINFLUX_CV&h._&i. LINFLUX_PARAMS&h._&i.;
DROP _MODEL_
_TYPE_
_DEPVAR_
_RMSE_
INTERCEPT
ETIM
CO2_uMOL
MODEL
DEPENDENT
LABEL1
CVALUE1
CVALUE2
NVALUE1
LABEL2
NVALUE2
VARIABLE
DF
ESTIMATE
STDERR;
BY LICORFILE OBS;
RUN;

DATA DATASET.LINEARFIT&h._&i.;
SET LINEARFIT&h._&i.;
RUN;

%END;
%END;
%MEND WINDOWS;
%WINDOWS;

RUN;

DATA ALL_LINEARFIT;
SET DATASET.LINEARFIT:;
BY LICORFILE OBS;
RUN;

DATA DATASET.ALL_LINEARFIT;
SET ALL_LINEARFIT;
RUN;

Re: proc reg error

Rick_SAS — Tue, 10 Oct 2017 13:29:14 GMT

But you can still use

ODS EXCLUDE ALL;

proc reg;

ODS OUTPUT TableName=DataSetName;

quit;

ODS EXCLUDE NONE;

to suppress output to the open ODS destinations (eg, HTML or LISTING)

Re: proc reg error

ninemileshigh — Tue, 10 Oct 2017 13:31:46 GMT

I am doing many regressions of LINFLUX against the same variable (ETIM), which is timeseries data, but moving the time window that the model starts for each observation, hence the loops

Re: proc reg error

ninemileshigh — Tue, 10 Oct 2017 13:32:53 GMT

thanks, I didn't know that