BookmarkSubscribeRSS Feed
Calcite | Level 5

I am trying to build a unique type of model scoring/comparison. Right now I am testing different models manually and I would like to automatically select which model performs best according to custom criteria.


Basically, I am building a dynamic pricing model, where 25% of users get one of A, B, C, or D prices. I then build one model for each price. So a model is run for everyone who got price A, and B, and so forth.


So then I have 4 models.


I then score all 4 models on my validation or test data and see which predicts the highest amount (response variable is revenue). Whichever model predicts the highest amount is my ending price segment. So if model A predicts the highest revenue for a customer, that customer is predicted to receive price A.


My model revenue is then the mean total revenue of all those customers who received the same price they were predicted to receive (or IOW all the customers who received the same price as the model that predicted the highest revenue)


I want to test different models to see which one predicts the highest model revenue. I can do it with sas code nodes manually, but I want to know if there is a way to do it automatically and select the highest models automatically.



I have learned that the start/end groups functionality wont work for my purposes since I cant ouput scores from each group with that functionality.



Any ideas? Thanks




Here is some help to show what I am doing:

Calculting uplift


Diagram of an example process (basically to show I am scoring the models separately. Each model is from a different subset of the data)


Super User

Have you looked at the Ensembke model node? 


Calcite | Level 5

From what I understand the ensemble node can take the average or maximum of different models from the same training data, correct? My goal is a bit different since I am trying to take the maximum of different sets of training data


Available on demand!

Missed SAS Innovate Las Vegas? Watch all the action for free! View the keynotes, general sessions and 22 breakouts on demand.


Register now!

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 2 replies
  • 2 in conversation