Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Home
- /
- Analytics
- /
- Forecasting
- /
- Simple analyses on unevenly-spaced timeseries data without converting ...

Options

- RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

🔒 This topic is **solved** and **locked**.
Need further help from the community? Please
sign in and ask a **new** question.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Posted 09-08-2016 05:29 PM
(2868 views)

I'm working with unevenly-spaced timeseries data from IoT devices. I'm new to SAS and had previously been using the Pyton traces package for handling this data.

One handy thing that traces enables is "simple analyses of unevenly-spaced time series data without making an awkward / lossy transformation to evenly-spaced representations." For example, if I wanted to know when any of a set of IoT devices were in an "on" state (do a logical OR), I could do that very easily with traces Timeseries objects - without first converting the data to a evenly-spaced representations and without writing any custom code to implement the logical or based on the uneven time steps.

Does SAS have any capabilities for directly performing simple analyses on unevenly-spaced time series data?

1 ACCEPTED SOLUTION

Accepted Solutions

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Here is the approach I would use to perform simple operations with base SAS coding:

```
/* Example data */
data t;
input timeStamp :datetime20. devId :$8. state;
format timestamp datetime17.;
datalines;
13SEP2016:01:01:00 A 0
13SEP2016:01:01:10 A 1
13SEP2016:01:01:11 A 0
13SEP2016:01:01:55 B 1
13SEP2016:02:20:00 A 1
13SEP2016:02:30:00 C 0
13SEP2016:02:33:00 B 0
13SEP2016:04:01:00 A 0
13SEP2016:05:01:00 C 1
13SEP2016:05:12:00 C 0
13SEP2016:05:23:00 C 1
13SEP2016:05:35:00 B 1
13SEP2016:06:00:00 A 1
13SEP2016:06:01:00 A 0
;
/* Get the list of devices */
ods output SQL_Results=devFmt;
proc sql number;
select unique devId, "devFmt" as fmtname, "I" as type
from t;
select cats("v_", Row, "=", quote(compress(devId)))
into :devList separated by " "
from devFmt;
quit;
proc format cntlin=devFmt(rename=(devId=start Row=label)); run;
/* Accumulate device states and perform simple operations */
data wide_t;
set t;
label &devList.;
retain v_:;
array _v v_:;
_v{input(devId, devFmt.)} = state;
OR_state = max(of v_:);
AND_state = min(of v_:);
NB_ON = sum(of v_:);
NB_OFF = n(of v_:) - NB_ON;
run;
proc print data=wide_t label noobs; run;
```

PG

10 REPLIES 10

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Note Specialized tools for time series are provided under the Forecasting and Econometrics (SAS/ETS) license. You might want to address your question to that Community

https://communities.sas.com/t5/SAS-Forecasting-and-Econometrics/bd-p/forecasting_econometrics

PG

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

It depends on your data structure.

Determining what's an ON state could be as simple as a filter and not require any special package.

Re unevenly spaced time series, I'll let someone else answer.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Hello -

SSM mastermind @rselukar point out to me that unevenly spaced time series are also called longitudinal data - there are 3 examples in PROC SSM documentation which deal with this type of situation:

- http://support.sas.com/documentation/cdl/en/etsug/68148/HTML/default/viewer.htm#etsug_ssm_examples04...
- http://support.sas.com/documentation/cdl/en/etsug/68148/HTML/default/viewer.htm#etsug_ssm_examples09...
- http://support.sas.com/documentation/cdl/en/etsug/68148/HTML/default/viewer.htm#etsug_ssm_examples15...

Furthermore he was kind enough to point me to PROC SSM documentation, that describes the types of sequential data SSM can handle:

And ouf course you may want to check out his SAS Global Forum paper: http://support.sas.com/resources/papers/proceedings15/SAS1580-2015.pdf on this very topic.

Many thanks Rajesh!

Udo

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

I was not aware of the traces package. Taking a quick look at the package link I realize that you are interested in more basic analysis of unevenly spaced timeseries. SSM procedure is designed for model based analysis of such timeseries and you can do rather sofisticated analysis of such data. Without going into too many details, I am going to explain how to use the SSM procedure to do basic interpolation/extrapolation of an unevenly spaced timeseries. Assume that your input data set, test, has two columns time and y. The the time column contains the times associated with the measurements y (the times need not be evenly spaced, in fact, you can even have multiple measurements at the same time point). It is also assumed that the data set test is sorted according time, and the time points at which you want interpolated/extrapolated values of y are included in the data set with corresponding y missing. So the first few rows of the data set might look something like this:

time y

1.2 .

1.8 -2.6

2.0 .

8.3 1.3

Note y values at time points 1.2 and 2.0 are missing. You can obtain a smooth interpolation of y values as follows:

proc ssm data=test;

id time;

trend scurve(ps(2)) checkbreak;

irregular noise;

model y = scurve noise / print=smooth;

output out=for press;

run;

The interpolated/extrapolated values of y are printed. SSM procedure stores the estimate of the smoothed curve (called smoothed_scurve) in the output data set. You can plot it and see the fit as follows:

proc sgplot data=for;

series x=time y=smoothed_scurve;

scatter x=time y=y;

run;

Note that you can specify many more interesting models (see the examples mentioned earlier), use predictor info if available, the CHECKBREAK option in the TREND statement identifies possible locations of abrupt changes in the smoothed curve, and lot more things.

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Thank you @udo_sas and @rselukar for this information about SSM! I see that it will be very helpful for extrapolation/interpolation, and also more powerful modeling of unevenly spaced timeseries data.

I'm still wondering if there is anything that will do simple arithmetic and logical operators on a set of unevenly-spaced timeseries data (such as what would come off of a set of related sensors). Each sensor will send data at different unevenly-spaced times. The example from the traces package website about adding up all the light switched on in a building (see below) is a good one, but I may also want to do different things like logical ORs to see when any light is on in the building, find the average number of lights on in the building, etc. Would SSM help with this? Or something else in SAS?

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Yes this is basically what I'm doing now, although with a DATA step instead of PROC TIMESERIES. I was hoping to avoid the somewhat awkward and potentially memory-intensive transition from unevenly-spaced to evenly spaced for simple arithmetic and logical computations, and then back to unevenly-spaced for further analysis.

Anyhow thanks for your help @rselukar !

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Here is the approach I would use to perform simple operations with base SAS coding:

```
/* Example data */
data t;
input timeStamp :datetime20. devId :$8. state;
format timestamp datetime17.;
datalines;
13SEP2016:01:01:00 A 0
13SEP2016:01:01:10 A 1
13SEP2016:01:01:11 A 0
13SEP2016:01:01:55 B 1
13SEP2016:02:20:00 A 1
13SEP2016:02:30:00 C 0
13SEP2016:02:33:00 B 0
13SEP2016:04:01:00 A 0
13SEP2016:05:01:00 C 1
13SEP2016:05:12:00 C 0
13SEP2016:05:23:00 C 1
13SEP2016:05:35:00 B 1
13SEP2016:06:00:00 A 1
13SEP2016:06:01:00 A 0
;
/* Get the list of devices */
ods output SQL_Results=devFmt;
proc sql number;
select unique devId, "devFmt" as fmtname, "I" as type
from t;
select cats("v_", Row, "=", quote(compress(devId)))
into :devList separated by " "
from devFmt;
quit;
proc format cntlin=devFmt(rename=(devId=start Row=label)); run;
/* Accumulate device states and perform simple operations */
data wide_t;
set t;
label &devList.;
retain v_:;
array _v v_:;
_v{input(devId, devFmt.)} = state;
OR_state = max(of v_:);
AND_state = min(of v_:);
NB_ON = sum(of v_:);
NB_OFF = n(of v_:) - NB_ON;
run;
proc print data=wide_t label noobs; run;
```

PG

- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content

Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.

**If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website. **

Multiple Linear Regression in SAS

Learn how to run multiple linear regression models with and without interactions, presented by SAS user Alex Chaplin.

Find more tutorials on the SAS Users YouTube channel.