HPFDiagnose Format

[Burch123]

Hello!

I was wondering if anyone can help!?

This HPFDIAGNOSE procedure is taking far too long and sometimes freezing completely. Have I coded something incorrectly? CPU and storage space are of no issue and all other operations run fine.

I have four datasets; essentially all to be passed through the HPFDIAGNOSE procedure.

1. The time series dataset has various products, all with about 1000 observations of sales (some missing, some present)

2. The Auxillary Dataset has Date (to be matched against the time series dataset) and trading hours as a continuous variable

3. The Event dataset contains about 300 events

4. The Event by dataset contains approx 20 events, by "PRODUCT_NUMBER"

These take the format of:

1. (Time Series Dataset)  XXXX.TIME_SERIES

PRODUCT_NUMBER (8.0)	DATE (DATE9.)	SALES (10.2)
1234	01MAR2014	7
1234	02MAR2014	8
1988844	23APR2014	12
...	...	...

2. (Trading Hours Dataset) XXXXX.TRADING_HOURS

DATE (DATE9.)	TRADING_HOURS(8.2)
01APR2009	6.7
02APR2009	7.2
03APR2009	8.1
04APR2009	7.6

3.(Event Dataset) XXXXXX.EVENT_DATASET (AS PER HPF EVENTS)

_NAME_	_CLASS_	_KEYNAME_	_STARTDATE_	...
PROMOTION	SIMPLE	.	01APR2009	...
PROMOTION	SIMPLE	.	07NOV2014	...
CHRISTMAS	SIMPLE	.	25DEC2014	...

4.(Event BY Dataset) XXXXXX.EVENT_DATASET_BY_PRODUCT

PRODUCT_NUMBER	_EVENT_	_NAME_	_REQUIRED_
1234	PROMOTION	SALES	YES(POSITIVE)
1234	CHRISTMAS	SALES	MAYBE

I've placed into the following:

%MACRO MODEL_SELECTION;

ARIMAX;

IDM;

UCM;

ESM;

%MEND MODEL_SELECTION;

PROC HPFDIAGNOSE

DATA=TIME_SERIES

MODELREPOSITORY=REPOSITORY_SAMPLE

AUXDATA=TRADING_HOURS

OUTEST=ESTIMATORS

OUTOUTLIER=OUTLIERS

INEVENT=EVENT_DATASET EVENTBY=EVENT_DATASET_BY_PRODUCT;

ID DATE INTERVAL=DAY;

FORECAST SALES;

INPUT TRADING_HOURS /REQUIRED=MAYBE;

BY PRODUCT_NUMBER;

%MODEL_SELECTION;

RUN;

Any help would be greatly appreciated!?

Kind Regards,

Chris

[udo_sas · Accepted solution]

Hello -

You may try without UCM and see if this improves performance. If you run into crashes (like freezing runs) you best bet is to contact Technical Support, as this might be data related.

Thanks,

Udo

[Burch123]

Udo,

Thanks for this, really great help.

I've just realised, I may have misinterpreted the INEVENT= and BYEVENT= functions and was wondering if you could help.

Suppose I have two time series to forecast, I only want the events in the BYEVENT part to be diagnosed, however the HPFDIAGNOSE procedure appears to be using all events from the INEVENT dataset?

For example

Suppose Time Series X & Y,

and INEVENT= is the below table

_NAME_	...
A	...
B	....
C	...
D	....
E	....
F	....
G	....

and BYEVENT= is the below table

Times Series	_EVENT_	_NAME_
X	A	SALES
X	B	SALES
X	C	SALES
Y	D	SALES
Y	E	SALES

So essentially, for time series X, I only want events A,B,C to be considered and subsequently for time series Y, D & E to be considered.

The way I have so far seems to be using all events for each "diagnose" loop for each "by" group.

My questions are:

     1) Have I structured something incorrectly?

     2) Is this how HPFDIAGNOSE works, and I have misinterpreted the procedure?

     3) If so, should I use a macro and call the events into the HPFDIAGNOSE procedure.

Any help appreciated as always!

Kind Regards.

[udo_sas]

Hello -

This example might be useful: https://communities.sas.com/message/52133#52133

Thanks,

Udo