Enterprise miner error msg when running HP Tree

[CathyGu]

when I was running the HP tree for a regression problem. The following error occurred with the error message of insufficient resources to proceed. Data size is about 156M records, with 70/30 split for train/validation. Please help. Thanks.

NOTE: External file /model/conr_out/qzrzct/New Folder/creditlossprediction/Workspaces/EMWS5/HPTree/EMFLOWSCORE.sas opened.

NOTE: External file /model/conr_out/qzrzct/New Folder/creditlossprediction/Workspaces/EMWS5/HPTree/ENGLISHRULES.txt opened.

NOTE: The HPSPLIT procedure is executing in single-machine mode.

ERROR: Insufficient resources to proceed.

NOTE: View WORK.HPTREE_TRAINDATA.VIEW used (Total process time):

      real time           27:23.02

      cpu time            26:19.49

NOTE: There were 111695660 observations read from the data set EMWS5.HPPART_TRAIN.

NOTE: There were 47869558 observations read from the data set EMWS5.HPPART_VALIDATE.

NOTE: The SAS System stopped processing this step because of errors.

NOTE: There were 159565218 observations read from the data set WORK.HPTREE_TRAINDATA.

WARNING: The data set EMWS5.HPTREE_VARIMP may be incomplete.  When this step was stopped there were 0 observations and 0 variables.

WARNING: Data set EMWS5.HPTREE_VARIMP was not replaced because this step was stopped.

WARNING: The data set EMWS5.HPTREE_OUTSUBTREE may be incomplete.  When this step was stopped there were 0 observations and 0 variables.

WARNING: Data set EMWS5.HPTREE_OUTSUBTREE was not replaced because this step was stopped.

WARNING: The data set EMWS5.HPTREE_TREEPLOT may be incomplete.  When this step was stopped there were 0 observations and 0 variables.

WARNING: Data set EMWS5.HPTREE_TREEPLOT was not replaced because this step was stopped.

WARNING: The data set EMWS5.HPTREE_OUTPRUNESTATS may be incomplete.  When this step was stopped there were 0 observations and 0 variables.

WARNING: Data set EMWS5.HPTREE_OUTPRUNESTATS was not replaced because this step was stopped.

NOTE: PROCEDURE HPSPLIT used (Total process time):

      real time           27:27.50

      cpu time            26:23.76

[RobWobDobBlobb]

Hi CathyGu,

NOTE: The HPSPLIT procedure is executing in single-machine mode.

ERROR: Insufficient resources to proceed.

Important to know about the HP-routines is that they are we're created with concurrent programming in mind (multiple cpus and/or threads executing in parallel).

From the note you can see that you're only running in single-machine mode, i.e. only one cpu is used. If you have more resources, check if miner has the correct access rights to those.

Insufficient resources can mean lack of two things: computing power or storage. Your 156M records most probably use most of the memory on your SAS Enterprise Miner server, which lets your HP Tree crash as it can not create any temporary intermediate tables (trees).

Have you tried loading a smaller data set (subset of 156M) and running the HP Tree?