turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- Analytics
- /
- Stat Procs
- /
- How to use WEIGHT in PROC QUANTREG?

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

08-15-2015 10:18 AM

PROC QUANTREG is not robust to high leverage points. Running:

proc quantreg data = sashelp.baseball plots = all;

model salary = YrMajor /quantile = (0.1, 0.5, 0.9);

run;

Gives a warning in the log that there are high leverage points and suggests using the WEIGHT statement. However, no details are available. I have seen some articles suggesting weighting by the inverse of leverage, but there isn't a lot of material.

How should this be done?

Peter

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to plf515

08-17-2015 08:25 AM

Hi Peter,

I poked at this a little bit, using the inverse of the robust distance as a weight. It appeared to reduce the number of outliers, but... The log says the leverage option still uses the unweighted values to generate the next step. I was hoping that some sort of iterative process could be applied, but no luck.

Steve Denham

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to SteveDenham

08-17-2015 08:35 AM

Thanks Steve. Hmm. Maybe the developer of QUANTREG will chime in.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to plf515

08-17-2015 10:34 AM

Use a log transformation on the response:

model logsalary = YrMajor /quantile = (**0.1**, **0.5**, **0.9**);

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Rick_SAS

08-17-2015 10:59 AM

Hi Rick

That does not solve the problem. There are still 26 leverage points and the warning is still given.

Peter

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to plf515

08-17-2015 11:39 AM

The WARNING is common from one of the plots. If you delete PLOTS=ALL, there is no warning.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Rick_SAS

08-17-2015 11:44 AM

Interesting! But that's surely a problem?

I know that QUANTREG is sensitive to high leverage points. But the extent of the sensitivity (and whether it is problematic) surely doesn't depend on whether one requests particular plots!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to plf515

08-17-2015 12:54 PM

SAS is efficient. If a statistic is not needed by an analysis, it is usually not computed. If you use the LEVERAGE option on the MODEL statement, you get the warning. If you specify PLOT=RDPLOT (which is part of PLOT=ALL), the proc needs to compute the leverage values, so it helps you out by specifying the option for you.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Rick_SAS

08-17-2015 01:07 PM

But why when a WEIGHT value is specified does the leverage estimation not change? To answer Peter's question, I would think that if you correctly specified weights for the observations then the leverage of given points with small weights would go down. I was hoping to set up a sort of iterative process that minimized the number of leverage points, but it doesn't look like the output dataset from a weighted analysis looks any different than an unweighted analysis. I suppose that is the result of the optimization method being used.

Steve Denham

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to SteveDenham

08-17-2015 02:23 PM

I'll be the first to agree that the word "weight" is used in a confusing fashion in the doc for the ROBUSTREG and QUANTREG docs. To me, it seems like the word is used in at least three different ways.

I think you need to look at the ROBUSTREG documentation, not the QUANTREG doc, to see how the WEIGHT statement is used to detect leverage points and outliers. The short answer is that the WEIGHT variable is ignored for that computation. Notice that the robust distances are documented as not being affected by the WEIGHT statement. Notice also that in ROBUSTREG, the "final weighted least squares estimates" refer to "the least squares estimates after the detected outliers are deleted." In other words, the OUTLIER indicator variable (0/1) is used as a weight to exclude observations.

Based on what I've just said, I don't really understand the warning message. You might want to take this up with Tech Support unless Steve has additional ideas. Maybe you are supposed to run the PROC once to get the leverage values (and the warning) and then you use that 0/1 variable to exclude the high-leverage points?:

proc quantreg data = sashelp.baseball ;

model logsalary = YrMajor /quantile=(0.1, 0.5, 0.9) leverage;

output out=out leverage=leverage;

run;

proc quantreg data = out;

weight leverage; /* exclude hihg-leverage from first run */

model logsalary = YrMajor /quantile=(0.1, 0.5, 0.9);

run;

Blah, blah, I'm not an expert, blah, blah, I don't speak for SAS, etc.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to Rick_SAS

08-17-2015 02:34 PM

Ok, time to talk to tech support.

I'll report back on what they say.