topic Re: Finding the inflection point with nlpfdd for best threshold in SAS/IML Software and Matrix Computations

Finding the inflection point for best threshold

harmonic — Mon, 10 Feb 2025 14:20:42 GMT

Hello community,

I would like to find the best threshold with a function like that:

For now I used the slop function to find the sign differences:

data sorted_QUANTILI_DS;
set sorted_QUANTILI_DS;
slope = dif(prob_nozero_cum) / dif(index); /* estimate the slope by using backward difference */
SlopeChanged = dif(slope); /* indicate when the slope changes */
run;

Is there a way to use the call nlpfdd based on this article and use the finite-differnece derivatives:
Finite-difference derivatives in SAS - The DO Loop

Thank you in advance

Re: Finding the inflection point with nlpfdd for best threshold

Rick_SAS — Mon, 27 Jan 2025 11:09:32 GMT

Yes, assuming that you have a formula for the function. If so, the inflection points are the values of x where the graph changes from concave up to concave down or vice versa. This requires finding a point where the second derivative changes sign, which means you can use the NLPFDD function to obtain numerical second derivatives. You can use the FROOT function in PROC IML to

find the numerical roots of any 1-D function on a specified interval. In the following, I find the inflection point for the function sin(x) when x is in the interval [pi,2, 3*pi/2]:

proc iml;
/* define the function */
start Func(x);
return( sin(x) );
finish;

/* define function that finds the numerical second derivative of 'Func' */
start SecondDeriv(x);
call nlpfdd(fun, deriv, second_deriv, 'Func', x);
return( second_deriv );
finish;

/* find the inflection point for Func on the interval [a,b] */
pi = constant('pi');
a = pi/2;
b = 3*pi/2;
infl_pt = froot("SecondDeriv", a||b );
print infl_pt;

For your example, the interval would be {0, 50}. You would replace the definition of the FUNC function with your function.

If you don't have a formula for the function, the problem gets much harder. You have to use some sort of interpolation scheme to approximate the function, then find the inflection point of the interpolant.

Re: Finding the inflection point with nlpfdd for best threshold

harmonic — Mon, 27 Jan 2025 11:46:53 GMT

Yes I do not have the form of the function, just the interval and the y. Do you have any advice for interpolation?

Re: Finding the inflection point with nlpfdd for best threshold

Rick_SAS — Mon, 27 Jan 2025 15:31:43 GMT

Sure. You say that you want a numerical estimate of the inflection point based only on knowing the function at a finite set of points (x[i], f(x[i])).

Here's what I suggest.

1. Use a finite-difference formula to approximate the second derivative of the function. I'd suggest using central differences for 2nd derivatives. Formulas are at
https://go.documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/imlug/imlug_nonlinearoptexpls_sect012.htm

You will get a vector (call if D2F) that for each value of x estimates the second derivative at x.

2. Find the location(s) where the second derivative changes sign.

3. Let i-1 and i be the indices for which D2F have different sign. Let xL = x[i-1] and xR = x[i], and let yL = D2F[i-1] and yR = D2f[i]. Then you can define
m = (yR - yL) / (xR - xL);

inflectionPt = xL - yL / m;

Re: Finding the inflection point with nlpfdd for best threshold

Rick_SAS — Mon, 10 Feb 2025 14:20:16 GMT

In summary, here are two references for how to numerically estimate inflection points:

1. Find an inflection point for a function when you know a formula (al algorithm) for the function

2. Find an inflection point when you know the function only at a finite set of points.

From what you've said, the second item is relevant to your problem.