topic Re: Results comparison between SAS Forecast Studio 15.3 and SAS Viya Model Studio pipeline in Moving to SAS Viya

Results comparison between SAS Forecast Studio 15.3 and SAS Viya Model Studio pipeline

devi001 — Tue, 14 Nov 2023 09:31:23 GMT

Hi All,

We are trying to compare results generated in SAS Forecast Studio 15.3 & SAS Viya Model Studio. We have observed that in Forecast Studio, Fit Statistic is MAPE where as in Viya Model Studio it is WMAPE. Please refer to the attached snapshots in the document.

Kindly assist on which Fit statistic we have to consider post migrating projects of SAS Forecast Studio 15.3 to SAS Viya Model Studio in order to maintain same accuracy in both Forecast studio and Viya Model Manager.

Please Note - Source dataset used in both projects is same which is SASHELP.PRICEDATA.

Thanks in advance!

Re: Results comparison between SAS Forecast Studio 15.3 and SAS Viya Model Studio pipeline

sbxkoenk — Tue, 14 Nov 2023 21:34:03 GMT

Hello,

Many people (including me) won't download and open *.docx files.

You speak about SAS Model Studio and SAS Model Manager.

But, actually, you have used VF (Visual Forecasting).

Model Studio is just an interface (it's not a SAS product in your license key).

Model Studio is the interface to:

VDMML (Visual Data Mining and Machine Learning)
VF (Visual Forecasting)
VTA (Visual Text Analytics)

BR, Koen

Re: Results comparison between SAS Forecast Studio 15.3 and SAS Viya Model Studio pipeline

sbxkoenk — Tue, 14 Nov 2023 21:39:59 GMT

Hello again,

Have you build up again a pipeline in VF (from scratch)?

Or did you use a "model migration"-tool from SAS (from FS to VF)?

Koen

Re: Results comparison between SAS Forecast Studio 15.3 and SAS Viya Model Studio pipeline

foresas — Tue, 14 Nov 2023 23:41:46 GMT

If you created far archive of the project on 9.4 and then imported it into Viya Model Studio then you will see MAPE in migrated HF node settings for model selection criterion. Running Forecast task should produce results as in FS project.

WMAPE is used in the Model Comparison down the pipeline.

It is used if you have more than one modeling node. With one node you will see: