Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Home
- /
- Analytics
- /
- Stat Procs
- /
- Change of slope in a curve

- RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Posted 11-01-2023 12:42 AM
(918 views)

Hi,

I have a clustered count data where I have counts and denominator within each cluster for a period of 80 months. In order to examine the effect of month on the rate I fitted a random intercept Poisson model. However, the relationship is very non-linear as shown in the plot of raw rates below.

Can you please suggest how can I identify the time points at which the slope of the curve is changing (turning points)?

22 REPLIES 22

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

I don't have a suggestion, but I think your question might be slightly different.

Is your question really "how can I fit a model to this sort of data?". Meaning if you could fit a model to it that you liked, then it would be straight forward to find the points where the slope of the model changes.

As shown in the blog post linked by @Ksharp , there are model-free ways to find local maxima and minima. I think in that context for this sort of data with some noise in it, you would need to think about how to empirically define the point of a slope change aka local min/max aka vertex (?). For example, perhaps think about how many "slope change" points you see on the plot, and then think backwards to a reasonable rule that would help identify those points.

Check out the Boston Area SAS Users Group (BASUG) video archives: https://www.basug.org/videos.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

By "turning point" do you mean "inflection point" ? https://en.wikipedia.org/wiki/Inflection_point

Inflection point has a standard usage but I am afraid "turning point" means nothing to me.

And if "month" is the desired "effect" on rate , not "month of year", then perhaps you need to look at things differently. And if the topic is "month" why does the graph use week for tick marks???

Several of the Timeseries related procedures deal with "seasonality" such as month in different years. Perhaps that is where you need to look.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Perhaps I misunderstood, but I was interpreting "turning point" as being a local minimum or maximum, where the sign of the slope changes. I haven't heard this term before, but an image on the wikipedia page you linked to uses "turning point" as a label for local minima and maxima.

Edit: I found a wikipedia page with a definition for turning point, consistent with my guess: "A **turning point** is a point at which the derivative changes sign."

https://en.wikipedia.org/wiki/Stationary_point

Check out the Boston Area SAS Users Group (BASUG) video archives: https://www.basug.org/videos.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

The term "turning point" may not be statistically appropriate. It should be "Change in trend". It is not only month but month of the year. I analysed data with both weekly and monthly resolution and by mistake I put the weekly graph. The monthly graph is given below.

During the period between 2018 and 2022 few intervention happened, whose exact timing is unclear, and they are supposed affect the trend of the rate of medication use. I need to identify those time points.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Can you highlight on the plot the points you would want to identify as "change in trend"? Just by eyeballing.

Check out the Boston Area SAS Users Group (BASUG) video archives: https://www.basug.org/videos.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

One possible solution is to define a piecewise linear regression model.

If you know the time at which the change occurs, you can solve the problem by using a piecewise linear spline and setting the "knot locations" to be the known times. See https://blogs.sas.com/content/iml/2017/04/05/nonsmooth-models-spline-effects.html

If you do not know the change points, then you have to solve for them. These are known as segmented regression models. For an example of a single change point, see https://blogs.sas.com/content/iml/2020/12/14/segmented-regression-sas.html

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

I do not know the time points when the changes occur and there are multiple of this. Is it possible to fit a Generalised Additive Model (GAM) to determine the smooth function and then find out the maximas and minimas by taking the derivatives?

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

- Paper SAS456-2017

Detecting and Adjusting Structural Breaks in Time Series and Panel Data

Using the SSM Procedure

Rajesh Selukar, SAS Institute Inc.

https://support.sas.com/resources/papers/proceedings17/SAS0456-2017.pdf

https://www.youtube.com/watch?v=7Mrmj-ERrM0 - The above paper also includes a brief review of the structural break detection facilities of other SAS/ETS procedures, such as the ARIMA, AUTOREG, and UCM.
- SAS/ETS Examples

Chow Test for Structural Breaks

https://support.sas.com/rnd/app/ets/examples/chow/index.htm

https://en.wikipedia.org/wiki/Chow_test - SAS/ETS® 15.1 User’s Guide

The UCM Procedure

https://support.sas.com/documentation/onlinedoc/ets/151/ucm.pdf (search term : CHECKBREAK) - Applying Data Science

by Gerhard Svolba

Chapter 6: Detecting Structural Changes in Longitudinal Data

Chapter 7 – Detecting Outliers and Level Shifts in Longitudinal Data

https://www.sas.com/storefront/aux/en/spba/63165_excerpt.pdf

I like using the ADAPTIVEREG Procedure for detecting breakpoints and structural breaks (see 6.3 !)

Br, Koen

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Rajesh gave an excellent presentation to BASUG last year on PROC SSM and PRO UCM. Slides and recording are in our archives: https://www.basug.org/archives

Check out the Boston Area SAS Users Group (BASUG) video archives: https://www.basug.org/videos.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

*> Is it possible to fit a GAM to determine the smooth function and then find out the maximas and minimas by taking the derivatives?*

Yes, it is possible, if that is what you want to do. If you evaluate any model (GAM, loess, spline, MARS,...) at points on a fine, evenly spaced grid for the independent variable, then you can use numerical differentiation to estimate the extrema. See Find points where a regression curve has zero slope - The DO Loop (sas.com)

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Hi,

Thanks for the link. Can we find the confidence interval of the identified points?

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

*> Can we find a confidence interval of the identified points?*

You could if you use a segmented regression model or some other rigorous statistical model. But if you fit some smoother to the data and find places where the derivative of the smoother is 0, I don't think you can get a CI automatically from the regression. For one thing, different smoothers will give you different result s. (For example, think about a loess smoother with different smoothing parameters!)

I think the fit-a-smoother-and-take-derivative method is a heuristic method that doesn't have much statistical theory behind it. If you want something like a confidence interval, you need a parameter in a statistical model.

Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.

**If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website. **

What is ANOVA?

ANOVA, or Analysis Of Variance, is used to compare the averages or means of two or more populations to better understand how they differ. Watch this tutorial for more.

Find more tutorials on the SAS Users YouTube channel.