topic Re: 2nd degree polynomial regression: Model is not full rank in SAS Programming

2nd degree polynomial regression: Model is not full rank

littlewho — Wed, 08 Jul 2020 09:58:50 GMT

Hello,

I am trying to run a 2nd degree regression on a dataset looking like the following:

	date	returns	continuous_date	continuous_dateSquared
1	2007Q4	-0.53	2007.75	4031060.0625

2008Q1

-5.11

2008

4032064

2008Q2

-3.176666667

2008.25

4033068.0625

2008Q3

-1.516666667

2008.5

4034072.25

2008Q4

-6.57

2008.75

4035076.5625

2009Q1

-8.495

2009

4036081

2009Q2

0.1066666667

2009.25

4037085.5625

2009Q3

6.475

2009.5

4038090.25

2009Q4

12.72

2009.75

4039095.0625

2010Q1

1.265

2010

4040100

2010Q2

-0.125

2010.25

4041105.0625

2010Q3

0.565

2010.5

4042110.25

2010Q4

2.495

2010.75

4043115.5625

The regression equation should be like the following: Returns(t) = a + b * time(t) + c * time(t)^2 and the code looks like this:

proc reg data=temp;
   model returns=continuous_date continuous_dateSquared;
   ods output ParameterEstimates=params;
run;

When I try to run this, I'm getting the following error:

That's strange, because using the regression option from sgplot, works just fine:

ods graphics / reset width=8in height=6in imagemap;
proc sgplot data=temp;
	reg x=continuous_date y=returns_avg / degree=2 nomarkers; 
	scatter x=continuous_date y=returns_avg; 
run;
ods graphics / reset;
title;

Could anyone point what I am doing wrong?

Thanks!

Re: 2nd degree polynomial regression: Model is not full rank

ballardw — Wed, 08 Jul 2020 15:19:29 GMT

Note that your SGPLOT does not include the squared date.

Also see the parameter for continuous_dateSquared of zero? Meaning that the parameter is not actually used:

y= ax**2 + bx + c is the same as y=bx+c when a=0.

Not that reg is the best tool but the magnitude of the values for your "dates" are pretty large compared to the values of returns and the squared version is doing about what I would expect because of the magnitude.

You might see something marginally more useful if subtract something like 2006 from "continuous_date" and then square that value (which will yield values more in the range of 1.5 to 100 instead of 4,000,000). So "continuous date" would interpret as "year from 2006" instead of the absolute year.

Re: 2nd degree polynomial regression: Model is not full rank

smantha — Wed, 08 Jul 2020 17:48:07 GMT

I see the following warnings in the code that supports Ballard's hypotheses

@littlewho wrote:

Hello,

I am trying to run a 2nd degree regression on a dataset looking like the following:

date returns continuous_date continuous_dateSquared

1
2007Q4 -0.53 2007.75 4031060.0625

2 2008Q1 -5.11 2008 4032064

3 2008Q2 -3.176666667 2008.25 4033068.0625

4 2008Q3 -1.516666667 2008.5 4034072.25

5 2008Q4 -6.57 2008.75 4035076.5625

6 2009Q1 -8.495 2009 4036081

7 2009Q2 0.1066666667 2009.25 4037085.5625

8 2009Q3 6.475 2009.5 4038090.25

9 2009Q4 12.72 2009.75 4039095.0625

10 2010Q1 1.265 2010 4040100

11 2010Q2 -0.125 2010.25 4041105.0625

12 2010Q3 0.565 2010.5 4042110.25

13 2010Q4 2.495 2010.75 4043115.5625

The regression equation should be like the following: Returns(t) = a + b * time(t) + c * time(t)^2 and the code looks like this:
proc reg data=temp;
   model returns=continuous_date continuous_dateSquared;
   ods output ParameterEstimates=params;
run;
When I try to run this, I'm getting the following error:

That's strange, because using the regression option from sgplot, works just fine:
ods graphics / reset width=8in height=6in imagemap;
proc sgplot data=temp;
	reg x=continuous_date y=returns_avg / degree=2 nomarkers; 
	scatter x=continuous_date y=returns_avg; 
run;
ods graphics / reset;
title;
Could anyone point what I am doing wrong?

Thanks!

WARNING: The range of variable cont_date is so small relative to its mean that there may be loss of accuracy in the computations.
You may need to rescale the variable to have a larger value of RANGE/abs(MEAN), for example, by using PROC STANDARD M=0;
WARNING: The range of variable cont_date_sq is so small relative to its mean that there may be loss of accuracy in the
computations. You may need to rescale the variable to have a larger value of RANGE/abs(MEAN), for example, by using PROC
STANDARD M=0;

Re: 2nd degree polynomial regression: Model is not full rank

littlewho — Thu, 09 Jul 2020 06:11:32 GMT

@ballardw wrote:
Note that your SGPLOT does not include the squared date.

I included the option "degree=2" in the sgplot reg instruction, shouldn't be this the way of including the squared date?

Thanks for the suggestion, I will try to rescale the parameters and see what I would get.