topic Re: Identify Zigzag point for timeseries in SAS Forecasting and Econometrics

Identify Zigzag point for timeseries

hhchenfx — Thu, 17 Nov 2022 21:40:42 GMT

Hi Everyone,

I want to find the zigzag point for timeseries and quite surprise, there isn't much discussion on this topic.

In the data below, the zigzag point should be number 10, 1 and 10

Can you please help me with the code?

The zigzag line is in picture below

https://www.google.com/imgres?imgurl=https%3A%2F%2Fi.stack.imgur.com%2FjphC7.png&imgrefurl=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F59559788%2Fpandas-zigzag-segmentation-of-data-based-on-local-minima-maxima&tbnid=t3U3wqE3BewlbM&vet=12ahUKEwi9mfXGlrb7AhV2mHIEHTumBacQMygLegUIARDKAQ..i&docid=Al5NbnEIPnABSM&w=1769&h=871&q=zigzag%20timeseries&ved=2ahUKEwi9mfXGlrb7AhV2mHIEHTumBacQMygLegUIARDKAQ

Thanks,

HHC

data have;
input value;
datalines;
6
7
8
8
10
9
8
8
8
7
6
8
8
4
5
1
2
3
4
6
6
8
9
10
6
8
6
5
1
;run;

data have; set have;
n=_N_;run;

Re: Identify Zigzag point for timeseries

PeterClemmensen — Thu, 17 Nov 2022 21:29:23 GMT

I have never heard of a Zigzag point. What is the logic? And what does your desired result look like here?

Re: Identify Zigzag point for timeseries

hhchenfx — Thu, 17 Nov 2022 21:41:16 GMT

I added a picture link in my original post.

Thanks,

HHC

Re: Identify Zigzag point for timeseries

PeterClemmensen — Thu, 17 Nov 2022 21:42:44 GMT

It does not explain the logic. Please be more specific about your requirements.

Re: Identify Zigzag point for timeseries

mkeintz — Fri, 18 Nov 2022 19:21:21 GMT

You want to effectively finds local minima and maxima in a time series. Those local extremes can then be connected to generate a set of zigzag line segments.

But you have to define for us

what "local" is, i.e. how large a window is required to examine for a change in direction, and
how large a change in direction must be to designate the local minimum or maximum.

Re: Identify Zigzag point for timeseries

hhchenfx — Fri, 18 Nov 2022 03:32:45 GMT

@mkeintz , you are 100% correct. The thing is for that Zigzag, normally other software have the "local" and "size of change" parameters and user can adjust.

Can you do something like that?

I find it quite weird that I can't find any code for this very popular timeseries measure 😕

Thanks,

HHC

Re: Identify Zigzag point for timeseries

Ksharp — Fri, 18 Nov 2022 11:51:28 GMT

calling @Rick_SAS

/*Here could give you a start*/
data have;
input value;
n+1;
datalines;
6
7
8
8
10
9
8
8
8
7
6
8
8
4
5
1
2
3
4
6
6
8
9
10
6
8
6
5
1
;

proc iml;
use have;
read all var _num_ into Pts[c=vname];
close;
 indices = cvexhull(Pts);            /* compute convex hull of the N points */
 hullIdx = indices[loc(indices>0)];  
 P = Pts[hullIdx, ];  
 call sort(p,2);
create p from p[c=vname];
append from p;
close;
quit;
data p2;
 set p;
 sign=sign(dif(value));
run;
data p3;
 set p2;
 by sign notsorted;
 if last.sign;
run;
data want;
 set have p3(rename=(n=_n value=_v));
run;

proc sgplot data=want;
series x=n y=value;
scatter x=_n y=_v/markerattrs=(symbol=starfilled color=red);
run;

Re: Identify Zigzag point for timeseries

Rick_SAS — Fri, 18 Nov 2022 13:58:44 GMT

I haven't heard the term "zigzag point" before, but it seems that you want to identify peaks and valleys in a noisy signal by using certain criteria about the height of the peaks and the width between peaks. In SAS Viya, you can use the PEAKLOC and PEAKINFO functions in SAS IML to identify peaks in a signal. These functions are not available in SAS 9.4.

SAS Event-Stream Processing also supports various methods for locating peaks.

Since this question appears related to time series, I will move it to a more appropriate community where experts like @rselukar might see it and provide additional assistance.

Re: Identify Zigzag point for timeseries

hhchenfx — Sat, 19 Nov 2022 01:22:21 GMT

Thank you, Ksharp.

I am digesting your code 😆

HHC

Re: Identify Zigzag point for timeseries

Ksharp — Sat, 19 Nov 2022 08:38:33 GMT

"In SAS Viya, you can use the PEAKLOC and PEAKINFO functions in SAS IML to identify peaks in a signal. These functions are not available in SAS 9.4. "

Disappointed . Need pay money for it .

Re: Identify Zigzag point for timeseries

hhchenfx — Sun, 20 Nov 2022 00:00:49 GMT

I might have to think about a way to include parameter to the code so I can provide more flexibility.

HHC

Re: Identify Zigzag point for timeseries

Ksharp — Sun, 20 Nov 2022 08:23:17 GMT

You can split this time series into many small series ,and use my code to get these points you want.

Or calling @Rick_SAS ,please let him wrote a IML code for you ?

Re: Identify Zigzag point for timeseries

Rick_SAS — Sun, 20 Nov 2022 11:48:58 GMT

LOL. I have a full-time job already, but thanks for thinking of me. Let's hear what the time series experts say.

Re: Identify Zigzag point for timeseries

rselukar — Mon, 21 Nov 2022 19:40:57 GMT

The responses given so far are quite good. I am not aware of any thing in SAS/ETS or (SAS/ECONOMETRICS) that is specifically designed for peak-discovery. I think this problem can also be called identifying turning points of a time series. For noisy time series the interest could be in finding the turning points of the underlying smooth signal. I will describe a quick way of doing this by using PROC UCM in SAS/ETS. Of course, this is a solution to a related problem and not the original one.

Let's say the value_t series can be thought of as a sum of mu_t, a smooth curve, and noise_t: value_t = mu_t + noise_t. The following code shows you how to fit this model when mu_t is modeled by an integrated random walk, which produces cubic smoothing spline, and Gaussian white noise:

proc ucm data=have;
id n interval=obs;
model value;
irregular;
level variance=0 noest;
slope plot=smooth;
forecast lead=0 outfor=for plot=decomp;
run;

The output data set "for" contains the estimates of mu_t and its slope (the columns s_level and s_slope). The turning points of mu_t could be located as places where the sign of adjacent s_level values change (i.e., points where s_level stops going upward/downward). These points are located by the following data steps:

data change;
set for(keep=n value s_level s_slope);
s1 = lag(s_slope);
turn = 0;
if n >= 2 then do;
if sign(s1) ^= sign(s_slope) then turn=1;
end;
run;

*The turns data set contains the discovered turn points: 5 17 24;

data turns;
set change;
where turn=1;
run;

The following plot shows how well s_level approximates the original series:

The following curve plots the slope (s_slope):

The following code shows the discovered turning points:

proc sgplot data=change;
title Plot of smoothed_value with identified turning points;
series x=n y=s_level;
refline 5 17 24/ axis=x;
run;

Of course, the identified turning points are not the turning points of the original series but of the underlying smooth signal. The following plot shows how they appear for the original curve:

proc sgplot data=change;
title Plot of value with identified turning points for smoothed_value;
series x=n y=value;
refline 5 17 24/ axis=x;
run;

I know this is a bit lengthy response.

Re: Identify Zigzag point for timeseries

Ksharp — Tue, 22 Nov 2022 04:36:03 GMT

I tested rselukar 's code and got this :

data have;
 set sashelp.stocks;
 if stock='Intel';
 keep date stock close;
run;
proc sort data=have;by date;run;
data have2;
 set have(keep=close rename=(close=value));
 n+1;
run;

proc ucm data=have2;
    id n interval=obs;
    model value;
    irregular;
    level variance=0 noest;
    slope plot=smooth;
    forecast lead=0 outfor=for plot=decomp;
run;



data change;
    set for(keep=n value s_level s_slope);
    s1 = lag(s_slope);
    turn = 0;
    if n >= 2 then do;
        if sign(s1) ^= sign(s_slope) then turn=1;
    end;
run;
data turns;
    set change;
    where turn=1;
run;
data all;
 set have2 turns(keep=n value rename=(n=_n value=_v));
run;
proc sgplot data=all;
title Plot of smoothed_value with identified turning points;
series x=n y=value;
scatter x=_n y=_v/markerattrs=(symbol=starfilled color=red);
run;

Re: Identify Zigzag point for timeseries

rselukar — Tue, 22 Nov 2022 15:45:48 GMT

By the way, as I mentioned earlier, the turning points are for the underlying smooth signal (s_level). You can slightly alter @jacksharp's code to get the plot of turning points of s_level as follows:

data all;
set change turns(keep=n s_level rename=(n=_n s_level=_v));
run;

proc sgplot data=all;
title Plot of smoothed_value with identified turning points;
series x=n y=s_level;
scatter x=_n y=_v/markerattrs=(symbol=starfilled color=red);
run;

Which results in the following plot:

Of course, @jacksharp's way of showing the turning points is much better than my earlier plots. Thanks!

Re: Identify Zigzag point for timeseries

hhchenfx — Tue, 22 Nov 2022 21:46:29 GMT

Thank you all for helping.

It is great code.

HHC