I am getting a warning message when running Proc quantreg:
"WARNING: Convergence was not attained. Duality gap 4.3888576E-8 could not be reduced further. Reduce the required accuracy or try a smaller step length with the KAPPA= option."
I do not understand this message. I guess that reducing the accuracy only takes the message away but does not change anything. I do not understand what reducing the step length with the KAPPA option does. Does it change the outcome or just the efficiency?
Is there an easy sollution to this problem? I would be greatful if someone could explain this warning message. Is it a severe warning?
I am not a world-class expert in this area, but from what you've told me I think it isn't surprising that the model for lower quantiles does not converge. An optimization algorithm will often "not converge" if there is not a unique optimum, and it sounds like this is the case here.
One option is to give up on constructing a model for the lower quantiles. The model does not make sense for the data.
Another option is to change the data by adding a small amount of random noise to the response variable. This would to overcome the dupicated value problem. Read the following articles that discuss quantiles for and GOF tests for rounded data. I also include an article about the smooth bootstrap, which shows how adding small random values can sometimes work around problems caused by rounded data:
- Binning data by quantiles? Beware of rounded values
Please post the SAS code that generates the error. Also read the documentation for the Interior Point method and the KAPPA= option. The error message appears to tell you the solution: decrease the step size. If you are using the defalt value of kappa ( who knows? No code posted!) you can try
proc quantreg algorithm=interior(kappa=0.8);
...
run;
or try ALGORITHM=IPM
Thank you for your answer. I am afraid that I do not fully understand the description of the Interior point algorithm. How would reducing the Kappa option effect the results?
Here is the code:
proc quantreg data=t_quantreg; model cP = var1 var2 var3 var4 / quantile= 0.10 to 0.90 by 0.10 PLOT=QUANTPLOT; run;
Did you try it? Did it work?
KAPPA is related to a step size during the optimization phase. When you optimize a nonlinear function, the algorithm starts with a guess and then takes steps that (hopefully) improve the guess. If the step size is too large, the algorithm might step over the optimum or bounce around without converging. If the step size is too small, the optimization will take a long time and many iterations to converge.
What happens if you try a different algorithm for estimating the parameters? For example, ALGORITHM=IPM or ALGORITHM=SMOOTH?
The smooth algorithm is not efficient enough. I stopped this procedure after about 14 hours. The ipm algorithm is not accepted. An error message says that the procedure expects interior, simplex or smooth.
Sounds like you are running an old release of SAS.
I would be interested in knowing whether you have many duplicate rounded values for the response variable. It could be the the nature of your data does not permit discrimination between the lower quantiles. For example, in the data Y={0, 0, 0, 0, 0, 0, 1, 2, 2, 3, 3}, the quantiles for 0.1, 0.2, 0.3, 0.4, and 0.5 are all the same (0). Consequently, there is no way to distinguish between those quantiles. Can you post a histogram of your response variable?
Also, how many observations in the data?
It is likely the case that proc quantreg cannot discriminate between quantiles. My data is skewed, have a lot of duplicates since it is derived from integers. The distribution looks like this.
And zooming in to the values between 1 and 5:
I have about 9.5 million observations.
I am not a world-class expert in this area, but from what you've told me I think it isn't surprising that the model for lower quantiles does not converge. An optimization algorithm will often "not converge" if there is not a unique optimum, and it sounds like this is the case here.
One option is to give up on constructing a model for the lower quantiles. The model does not make sense for the data.
Another option is to change the data by adding a small amount of random noise to the response variable. This would to overcome the dupicated value problem. Read the following articles that discuss quantiles for and GOF tests for rounded data. I also include an article about the smooth bootstrap, which shows how adding small random values can sometimes work around problems caused by rounded data:
- Binning data by quantiles? Beware of rounded values
Thank you Rick! Now I understand the problem and can proceed with my work. One last question; Do you know how the procedure deals with the problem? Since I get a result it has delt with the problem somehome. In the data Y={0,1,2,2,3,4,5,6,7,8}. The first decile is 0 and the second decile 1 but what about decile 3 and 4, are the values arbitrarily assigned?
As discussed in the "Binning data by quantiles" article, the 30th pctl is 2 and the 40th pctl is also 2. Quantiles are not unique when you have repeated values.
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.
Find more tutorials on the SAS Users YouTube channel.