BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
MerceraS
Fluorite | Level 6

Good afternoon,

 

I'm trying to run two decision tree models using the bagging and boosting capabilities in SAS Enterprise Miner 14.2 using the Start Groups and End Groups, not the Ensemble node. Every time I run the models I receive an error that says "Run time error was encountered. The system error returned was 1012." I can't find anything in the SAS script that tells me anymore information than this about what went wrong. Has anyone else encountered this problem? What is the solution for it? I've been working on this since the weekend and I can't determine the source of the problem. My assignment is due at midnight on August 2 so help is needed urgently.

 

Thank you in advance.

*------------------------------------------------------------*
User:                mercep830
Date:                August 01, 2017
Time:                16:40:41
Site:                70094220
Platform:            Linux
Maintenance Release: 9.04.01M4P110916
EM Version:          14.2
*
*------------------------------------------------------------*
* Training Log
Date:                August 01, 2017
Time:                16:40:40
*------------------------------------------------------------*
15253  proc freq data=EMWS3.Grp2_VariableSet noprint;
15254  table ROLE*LEVEL/out=WORK.Grp2META;
15255  run;
 
NOTE: There were 2 observations read from the data set EMWS3.GRP2_VARIABLESET.
NOTE: The data set WORK.GRP2META has 2 observations and 4 variables.
NOTE: PROCEDURE FREQ used (Total process time):
      real time           0.00 seconds
      user cpu time       0.01 seconds
      system cpu time     0.00 seconds
      memory              22932.75k
      OS Memory           31608.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  0
      Page Faults                       0
      Page Reclaims                     643
      Page Swaps                        0
      Voluntary Context Switches        1
      Involuntary Context Switches      0
      Block Input Operations            0
      Block Output Operations           1320
 
 
15256  proc print data=WORK.Grp2META label noobs;
15257  var ROLE LEVEL COUNT;
15258  label ROLE = "%sysfunc(sasmsg(sashelp.dmine, meta_role_vlabel, NOQUOTE))" LEVEL = "%sysfunc(sasmsg(sashelp.dmine, meta_level_vlabel, NOQUOTE))" COUNT = "%sysfunc(sasmsg(sashelp.dmine, rpt_count_vlabel, NOQUOTE))";
15259  title9 ' ';
15260  title10 "%sysfunc(sasmsg(sashelp.dmine, rpt_varSummary_title  , NOQUOTE))";
15261  run;
 
NOTE: There were 2 observations read from the data set WORK.GRP2META.
NOTE: The PROCEDURE PRINT printed page 1.
NOTE: PROCEDURE PRINT used (Total process time):
      real time           0.00 seconds
      user cpu time       0.00 seconds
      system cpu time     0.01 seconds
      memory              22932.75k
      OS Memory           31608.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  0
      Page Faults                       0
      Page Reclaims                     736
      Page Swaps                        0
      Voluntary Context Switches        0
      Involuntary Context Switches      0
      Block Input Operations            0
      Block Output Operations           0
 
 
15262  title10;
 
15263  %let EMEXCEPTIONSTRING=;
15574  *------------------------------------------------------------*;
15575  * Grp2: Generation of macros and macro variables;
15576  * To see the code generated, set the EM_DEBUG macro variable to SOURCE or _ALL_;
15577  *------------------------------------------------------------*;
 
15578  %let EMEXCEPTIONSTRING=;
15579  *------------------------------------------------------------*;
15580  * TRAIN: Grp2;
15581  *------------------------------------------------------------*;
15582  %let EM_ACTION = TRAIN;
15583  %let syscc = 0;
15584  %macro main;
15585
15586     filename temp catalog 'sashelp.emutil.gp_macros.source';
15587     %include temp;
15588     filename temp;
15589
15590     %SetProperties;
15591
15592     %if %upcase(&EM_ACTION) = CREATE %then %do;
15593
15594         filename temp catalog 'sashelp.emutil.gp_create.source';
15595         %include temp;
15596         filename temp;
15597
15598         %create;
15599     %end;
15600     %else
15601     %if %upcase(&EM_ACTION) = TRAIN %then %do;
15602
15603         filename temp catalog 'sashelp.emutil.gp_train.source';
15604         %include temp;
15605         filename temp;
15606
15607         %train;
15608     %end;
15609     %else
15610     %if %upcase(&EM_ACTION) = SCORE %then %do;
15611
15612         filename temp catalog 'sashelp.emutil.gp_score.source';
15613         %include temp;
15614         filename temp;
15615
15616         %score;
15617     %end;
15618     %else
15619     %if %upcase(&EM_ACTION) = REPORT %then %do;
15620
15621         filename temp catalog 'sashelp.emutil.gp_report.source';
15622         %include temp;
15623         filename temp;
15624
15625         %report;
15626
15627     %end;
15628     %else
15629     %if %upcase(&EM_ACTION) = POSTLOOP %then %do;
15630
15631         filename temp catalog 'sashelp.emutil.gp_postloop.source';
15632         %include temp;
15633         filename temp;
15634
15635         %postloop;
15636
15637     %end;
15638
15639     %doendm:
15640  %mend main;
15641
15642  %main;
NOTE: %INCLUDE (level 1) file TEMP is file SASHELP.EMUTIL.GP_MACROS.SOURCE.
15643 +%macro SetProperties;
15644 +   %em_checkmacro(name=EM_PROPERTY_MODE,            global=Y, value=STRATIFY);
15645 +   %em_checkmacro(name=EM_PROPERTY_TARGETGROUP,     global=Y, value=Y);
15646 +   %em_checkmacro(name=EM_PROPERTY_INDEXCOUNT,      global=Y, value=10);
15647 +   %em_checkmacro(name=EM_PROPERTY_MINIMUMOBS,      global=Y, value=10);
15648 +   %em_checkmacro(name=EM_PROPERTY_SIZETYPE,        global=Y, value=PERCENT);
15649 +   %em_checkmacro(name=EM_PROPERTY_SIZEOBS,         global=Y, value=.);
15650 +   %em_checkmacro(name=EM_PROPERTY_SIZEPERCENT,     global=Y, value=10);
15651 +%mend SetProperties;
15652 +
NOTE: %INCLUDE (level 1) ending.
NOTE: Fileref TEMP has been deassigned.
NOTE: %INCLUDE (level 1) file TEMP is file SASHELP.EMUTIL.GP_TRAIN.SOURCE.
15653 +%macro scoreValidTest(data=, out=);
15655 +   %if ("&data" ne "") and (%sysfunc(exist(&data, VIEW)) or %sysfunc(exist(&data))) %then %do;
15656 +       data &out / view = &out;
15657 +          retain _RESAMP_ 1;
15658 +          set &data;
15659 +       run;
15660 +   %end;
15662 +%mend scoreValidTest;
15664 +%macro getVariables;
15665 +   %if "&emloopnumber" = "1" %then %do;
15666 +       data _null_;
15667 +          retain numgoup 0;
15668 +          set &em_data_variableset end=eof;
15669 +          %if &em_num_target = 1 %then %do;
15670 +              %if &EM_PROPERTY_MODE = BOOSTING %then %do;
15671 +                  where (role = 'TARGET' and level ^= 'INTERVAL' and grouprole in('BOOST', 'DEFAULT'));
15672 +                  if role eq 'TARGET' then call symput('emboostvar', strip(NAME));
15673 +              %end;
15674 +              %else
15675 +              %if &EM_PROPERTY_MODE = BAGGING %then %do;
15676 +                  where (role = 'TARGET' and grouprole in('BAG', 'DEFAULT'));
15677 +                  if role eq 'TARGET' then call symput('embagvar', strip(NAME));
15678 +              %end;
15679 +          %end;
15680 +          %else %do;
15681 +              where (grouprole= 'GROUP' or (role = 'TARGET' and grouprole in('BAG', 'BOOST')) or (role = 'SEGMENT' and grouprole = 'DEFAULT'));
15682 +              if grouprole eq 'BAG' then call symput('embagvar', strip(NAME));
15683 +              else
15684 +              if grouprole eq 'BOOST' then call symput('emboostvar', strip(NAME));
15685 +          %end;
15686 +          if grouprole = 'GROUP' or (role = 'SEGMENT' and grouprole = 'DEFAULT') then do;
15687 +             call symput('emgroupvar', strip(symget('emgroupvar'))!!' '!!strip(NAME));
15688 +             numgroup + 1;
15689 +          end;
15690 +          if eof then call symput('emnumgroupvar', strip(put(numgroup, best.)));
15691 +       run;
15692 +    %end;
15693 +    %else %do;
15694 +          %if "&EM_PROPERTY_MODE" = "STRATIFY" or "&EM_PROPERTY_MODE" = "CROSSVALIDATION"  %then %do;
15695 +            data _null_;
15696 +               retain numgoup 0;
15697 +               set &em_data_variableset end=eof;
15698 +               where (grouprole= 'GROUP' or (role = 'SEGMENT' and grouprole = 'DEFAULT'));
15699 +               call symput('emgroupvar', strip(symget('emgroupvar'))!!' '!!strip(NAME));
15700 +               numgroup + 1;
15701 +               if eof then call symput('emnumgroupvar', strip(put(numgroup, best.)));
15702 +            run;
15703 +          %end;
15704 +            %else
15705 +        %if &EM_PROPERTY_MODE = BOOSTING %then %do;
15706 +            %em_getName(key=BOOSTINFO,   type=DATA);
15707 +            %if %sysfunc(exist(&em_user_boostinfo)) %then %do;
15708 +                data _null_;
15709 +                   set &em_user_boostinfo;
15710 +                   call symput('emboostvar', strip(_TARGET_));
15711 +                run;
15712 +            %end;
15713 +        %end;
15714 +        %else
15715 +            %if &EM_PROPERTY_MODE = BAGGING %then %do;
15716 +                %em_getName(key=BAGINFO,   type=DATA);
15717 +                %if %sysfunc(exist(&em_user_baginfo)) %then %do;
15718 +                    data _null_;
15719 +                       set &em_user_baginfo;
15720 +                      call symput('embagvar', strip(_TARGET_));
15721 +                    run;
15722 +                %end;
15723 +            %end;
15724 +    %end;
15725 +%mend getVariables;
15727 +%macro gp_bag(targetname=);
15728 +   %if %index(&EM_DEBUG, _ALL_) %then %do;
15729 +        %put LOOP MODE:   &emloopmode ;
15730 +        %put LOOP NUMBER: &emloopnumber;
15731 +        %put BAGGING TARGET: &targetname;
15732 +   %end;
15734 +   filename temp catalog 'sashelp.emutil.gp_boostmacros.source';
15735 +   %include temp;
15736 +   filename temp;
15738 +   %em_getName(key=BAGINFO,    type=DATA);
15739 +   %em_getName(key=EMLOOPINFO, type=DATA);
15741 +   %if &emloopnumber = 1 %then %do;
15742 +       filename gpDelta "&EM_FILE_CDELTA_TRAIN";
15743 +       data _null_;
15744 +          file gpdelta;
15745 +          put "if ROLE = 'TARGET' and NAME ne '&targetName' then delete;";
15746 +          put "else if upcase(NAME) eq '_RESAMP_' then do;";
15747 +          put "     LABEL='Bootstrap Frequency'; LEVEL='INTERVAL'; ROLE='FREQ';";
15748 +          put "end;";
15749 +       run;
15750 +       filename gpdelta;
15752 +       %if %sysfunc(exist(&em_user_baginfo)) %then %do;
15753 +           proc datasets lib=&em_lib nolist;
15754 +              delete &em_nodeid._baginfo;
15755 +           run;
15756 +       %end;
15758 +       %if "%em_freq" ne "" %then %do;
15759 +           proc means data=&em_import_data sum;
15760 +              var %em_freq;
15761 +              output out = _tempds sum=sumfreq;
15762 +           run;
15763 +           quit;
15764 +           data _null_;
15765 +              set _tempDs;
15766 +              call symput('_sumfreq', put(sumfreq, best12.));
15767 +           run;
15768 +       %end;
15769 +       %else %do;
15770 +           proc sql noprint;
15771 +              select count(*) into :_sumfreq
15772 +              from &em_import_data;
15773 +           quit;
15774 +       %end;
15776 +       %if "&em_property_SizeType" eq "PERCENT" %then %do;
15777 +           data _null_;
15778 +              call symput('_size', put(round(&_sumfreq*&em_property_sizepercent/100), best12.));
15779 +           run;
15780 +       %end;
15781 +       %else
15782 +           %let _size = &em_property_sizeobs;
15784 +       data &em_user_baginfo;
15785 +          length _LOOP_ 8 _TARGET_ $32 _SUMFREQ_ 8 _SIZE_ 8;
15786 +          _LOOP_     =&EM_PROPERTY_INDEXCOUNT;
15787 +          _TARGET_   ="&targetname";
15788 +          _SUMFREQ_  =&_sumfreq;
15789 +           _SIZE_    = &_size;
15790 +          output;
15791 +       run;
15792 +       data &em_user_emloopinfo;
15793 +          length MODE $8;
15794 +          label MODE     = "%sysfunc(sasmsg(sashelp.dmine, rpt_mode_vlabel,       NOQUOTE))"
15795 +                _LOOP_   = "%sysfunc(sasmsg(sashelp.dmine, rpt_groupIndex_vlabel, NOQUOTE))"
15796 +               _TARGET_ = "%sysfunc(sasmsg(sashelp.dmine, rpt_target_vlabel,     NOQUOTE))";
15797 +           set &em_user_baginfo(keep=_LOOP_ _TARGET_);
15798 +           MODE='Bagging';
15799 +       run;
15801 +       data &em_data_eminfo;
15802 +          length TARGET KEY $32  DATA $43;
15803 +          TARGET=''; KEY='BAGINFO'; DATA="&em_nodeid"; output;
15804 +       run;
15805 +   %end;
15806 +   %else %do;
15807 +      data _null_;
15808 +         set &em_user_baginfo;
15809 +         call symput('_sumfreq', put(_sumfreq_, best12.));
15810 +         call symput('_size', put(_size_, best12.));
15811 +      run;
15812 +   %end;
15814 +   %let bagsumFreq  = &_sumFreq;
15815 +   %let bagSize     = &_Size;
15817 +   %embag( _tra=&em_export_train, _train=&em_import_data, _seed=&em_property_randomseed, _freq=%em_freq,
15818 +           _sumfreq=&bagsumfreq, _size=&bagsize, _loop=&emloopnumber);
15820 +   %scoreValidTest(data=&em_import_validate, out=em_export_validate);
15821 +   %scoreValidTest(data=&em_import_test,     out=em_export_test);
15823 +   %doendbg:
15825 +%mend gp_bag;
15828 +%macro gp_boost(targetname=);
15829 +   %if %index(&EM_DEBUG, _ALL_) %then %do;
15830 +       %put LOOP MODE:   &emloopmode ;
15831 +       %put LOOP NUMBER: &emloopnumber;
15832 +        %put BAGGING TARGET: &targetname;
15833 +   %end;
15835 +   filename temp catalog 'sashelp.emutil.gp_boostmacros.source';
15836 +   %include temp;
15837 +   filename temp;
15839 +   %em_getname(key=EMLOOPINFO, type=DATA);
15840 +   %em_getname(key=BOOSTWEIGHT, type=DATA);
15841 +   %em_getname(key=BOOSTSCORE,  type=DATA);
15842 +   %em_getName(key=BOOSTINFO,   type=DATA);
15844 +   %if &emloopnumber = 1 %then %do;
15845 +       filename gpDelta "&EM_FILE_CDELTA_TRAIN";
15846 +       data _null_;
15847 +          file gpdelta;
15848 +          put "if ROLE = 'TARGET' and NAME ne '&targetName' then delete;";
15849 +          put "else if upcase(NAME) eq '_RESAMP_' then do;";
15850 +          put "     LABEL='Arcing Frequency'; LEVEL='INTERVAL'; ROLE='FREQ';";
15851 +          put "end;";
15852 +       run;
15853 +       filename gpdelta;
15855 +       %let deleteString =;
15856 +       %if %sysfunc(exist(&em_user_boostweight)) %then %let deleteString = &deleteString &em_nodeid._boostweight;
15857 +       %if %sysfunc(exist(&em_user_boostscore))  %then %let deleteString = &deleteString &em_nodeid._boostscore;
15858 +       %if "&deleteString" ne "" %then %do;
15859 +           proc datasets lib=&em_lib nolist;
15860 +              delete &deleteString;
15861 +           run;
15862 +       %end;
15864 +       %emboost0( _train=&em_import_data, _dmboods=&em_user_boostweight, _target=&targetname, _freq=%em_freq);
15866 +       data &em_user_boostinfo;
15867 +          length _LOOP_ 8 _TARGET_ $32 _DMBOOST_DIVISOR_ 8 _SUMFREQ_ 8;
15868 +          _LOOP_            =&EM_PROPERTY_INDEXCOUNT;
15869 +          _TARGET_          ="&targetname";
15870 +          _DMBOOST_DIVISOR_ =&_dmboodiv;
15871 +          _SUMFREQ_         =&_sumfreq;
15872 +       run;
15873 +       data &em_user_emloopinfo;
15874 +          length MODE $8;
15875 +          label MODE     = "%sysfunc(sasmsg(sashelp.dmine, rpt_mode_vlabel,       NOQUOTE))"
15876 +                _LOOP_   = "%sysfunc(sasmsg(sashelp.dmine, rpt_groupIndex_vlabel, NOQUOTE))"
15877 +               _TARGET_ = "%sysfunc(sasmsg(sashelp.dmine, rpt_target_vlabel,     NOQUOTE))";
15878 +           set &em_user_boostinfo(keep=_LOOP_ _TARGET_);
15879 +           MODE='Boosting';
15880 +       run;
15882 +       data &em_data_eminfo;
15883 +          length TARGET KEY $32  DATA $43;
15884 +          TARGET=''; KEY='BOOSTINFO'; DATA="&em_nodeid"; output;
15885 +       run;
15887 +   %end;
15888 +   %else %do;
15889 +       %global _dmboodiv;
15890 +       %let dmboosf = %upcase(&targetname);
15891 +       %if %sysfunc(length(&targetname))> 30 %then
15892 +           %let dmboosf =  %substr(&targetname, 1, 30);
15894 +       data _null_;
15895 +          set &em_user_boostinfo;
15896 +          call symput('_SUMFREQ',  strip(put(_sumfreq_,best12.)));
15897 +       run;
15899 +       %emboost2(_freq=%em_freq, _dmboods=&em_user_boostweight, _dmensds=&em_user_boostscore, _dmboosf=&dmboosf, _sumfreq=&_sumfreq);
15901 +       data &em_user_boostinfo;
15902 +          set &em_user_boostinfo;
15903 +          _DMBOOST_DIVISOR_ =&_dmboodiv;
15904 +       run;
15906 +       %emboost1(_tra=&em_export_train, _TRAIN=&em_import_data, _FREQ=%em_freq, _dmboods=&em_user_boostweight, _dmboodiv=&_dmboodiv);
15908 +   %end;
15910 +    %scoreValidTest(data=&em_import_validate, out=em_export_validate);
15911 +    %scoreValidTest(data=&em_import_test,     out=em_export_test);
15913 +   %doendbst:
15915 +%mend gp_boost;
15917 +%macro gp_group(numgroupvar=, groupvar=);
15918 +   %em_getname(key=GROUPINFO, type=DATA);
15919 +   %em_getName(key=LOOPINFO,  type=DATA);
15920 +   %em_getName(key=EMGROUPINFO, type=DATA);
15922 +   %if &emloopnumber=1 or ^%sysfunc(exist(&em_user_groupinfo)) %then %do;
15923 +       ods listing close;
15924 +       %if &numgroupvar = 1 %then %do;
15925 +           ods output onewayfreqs=temp;
15926 +       %end;
15927 +       %else %do;
15928 +           ods output crosstabfreqs=temp;
15929 +       %end;
15930 +       proc freq data=&em_import_data;
15931 +          table
15933 +          %let tableString =;
15934 +          %let freqString  =;
15935 +          %do i=1 %to &numgroupvar;
15936 +              %let tableString = &tableString %scan(&groupvar, &i, %str( ));
15937 +              %if &i<&numgroupvar %then %let tableString = &tableString *;
15938 +              %let freqString = &freqString.1;
15939 +          %end;
15940 +          &tableString / missing;
15941 +       run;
15942 +       ods listing;
15943 +       %if &numgroupvar = 1 %then %do;
15944 +           data _null_;
15945 +              retain totalobs 0;
15946 +              set temp end=eof;
15947 +              totalobs + frequency;
15948 +              if eof then
15949 +                 call symput('_totalObs_', put(totalobs, best.));
15950 +           run;
15952 +           proc sort data=temp out=loopinfo nodupkey;
15953 +              by &groupvar;
15954 +              where frequency>=&em_property_minimumObs;
15955 +           run;
15956 +       %end;
15957 +       %else %do;
15958 +           data _null_;
15959 +              set temp;
15960 +              where _type_='00';
15961 +              call symput('_totalObs_', put(frequency, best.));
15962 +           run;
15963 +           proc sort data=temp(drop=rowpercent colpercent) out=loopinfo nodupkey;
15964 +              by &groupvar;
15965 +              where _type_="&freqString" and frequency>=&em_property_minimumObs;
15966 +           run;
15967 +       %end;
15969 +       %em_getname(key=PRECODE, type=FILE, extension=sas);
15970 +       filename _pregrp "&em_user_precode";
15972 +       %let dsid = %sysfunc(open(loopinfo));
15973 +       %do i=1 %to &numgroupvar;
15974 +           %let varname = %scan(&groupvar, &i, %str( ));
15975 +           %let varnum  = %sysfunc(varnum(&dsid, &varname));
15976 +           %let varfmt  = %sysfunc(varfmt(&dsid, &varnum));
15977 +           %if %index(&varfmt, -) %then %do;
15978 +               %let varfmt = %sysfunc(scan(&varfmt, 1,-)).;
15979 +           %end;
15981 +           %let vartype = %sysfunc(vartype(&dsid, &varnum));
15982 +           %let varlen  = %sysfunc(varlen(&dsid, &varnum));
15983 +           %let fmttype&i = &vartype;
15984 +           %let fmtlen&i  = &varlen;
15986 +           %let fmtvar&i =;
15987 +           %let fmt&i    =;
15988 +           %if "&varfmt" ne "" %then %do;
15989 +              %let fmtvar&i  = _FORMAT&i._;
15990 +              %let fmt&i     = &varfmt;
15991 +           %end;
15992 +       %end;
15993 +       %let dsid = %sysfunc(close(&dsid));
15995 +       data _null_;
15996 +          %do i=1 %to &numgroupvar;
15997 +              %if "&&fmt&i" ne "" %then %do;
15998 +                  retain len_&i 0;
15999 +              %end;
16000 +          %end;
16002 +          set loopinfo end=eof;
16003 +          file _pregrp;
16005 +          %do i=1 %to &numgroupvar;
16006 +              %let varname = %scan(&groupvar, &i, %str( ));
16007 +              %if "&&fmt&i" ne "" %then %do;
16008 +                 if length(put(&varname, &&fmt&i)) > len_&i then
16009 +                    len_&i = length(put(&varname, &&fmt&i));
16010 +              %end;
16011 +          %end;
16013 +          if eof then do;
16014 +             %do i=1 %to &numgroupvar;
16015 +                 %let varname = %scan(&groupvar, &i, %str( ));
16016 +                 %if "&&fmt&i" ne "" %then %do;
16017 +                     put "length  &&fmtvar&i" len_&i "$" len_&i ";";
16018 +                     put "drop &&fmtvar&i" len_&i ";";
16019 +                     put "&&fmtvar&i" len_&i "= strip(put(&varname, &&fmt&i));";
16020 +                     call symput('fmtnewlen'!!"&i", put(len_&i, best.));
16021 +                     call symput('fmtnewvar'!!"&i", "&&fmtvar&i"!!strip(put(len_&i, best.)));
16022 +                 %end;
16023 +              %end;
16024 +          end;
16025 +      run;
16027 +      data loopinfo;
16028 +         set loopinfo;
16029 +         %do i=1 %to &numgroupvar;
16030 +             %let varname = %scan(&groupvar, &i, %str( ));
16031 +             %if "&&fmt&i" ne "" %then %do;
16032 +                 length  &&fmtnewvar&i $&&fmtnewlen&i;
16033 +                 &&fmtnewvar&i = strip(put(&varname, &&fmt&i));
16034 +             %end;
16035 +         %end;
16036 +      run;
16037 +      filename _pregrp;
16039 +      %if (&EM_PROPERTY_TARGETGROUP= Y and &EM_NUM_TARGET>1) %then %do;
16040 +          data temp;
16041 +             length _LOOP_ 8 _WHERE_  _WHEREDESC_ $2000;
16042 +             keep _LOOP_ _WHERE_ _WHEREDESC_ FREQUENCY;
16043 +      %end;
16044 +      %else %do;
16045 +          data &em_user_groupinfo;
16046 +             length _LOOP_ 8 _TARGET_ $32 _WHERE_  _WHEREDESC_ $2000;
16047 +             keep _LOOP_ _TARGET_  _WHERE_ _WHEREDESC_ FREQUENCY;
16048 +             retain _TARGET_ '';
16049 +      %end;
16051 +          set loopinfo;
16052 +          _LOOP_= _N_;
16053 +          _WHERE_ = '';
16054 +          _WHEREDESC_  = '';
16055 +          %do i=1 %to &numgroupvar;
16056 +              %let varname = %scan(&groupvar, &i, %str( ));
16057 +              %if "&&fmt&i" eq "" %then %do;
16058 +                  %if "&&fmtType&i" eq "N" %then %do;
16059 +                      _WHERE_     = strip(_WHERE_)!!" &Varname ="!!strip(put(&varname, best12.));
16060 +                      _WHEREDESC_ = strip(_WHEREDESC_)!!" &Varname ="!!strip(put(&varname, best12.));
16061 +                   %end;
16062 +                  %else %do;
16063 +                       _WHERE_ = strip(_WHERE_)!!" &Varname ='"!!tranwrd(strip(&varname), "'", "''")!!"'";
16064 +                       _WHEREDESC_ = strip(_WHEREDESC_)!!" &Varname ="!!strip(&varname);
16065 +                  %end;
16066 +              %end;
16067 +              %else %do;
16068 +                       _WHERE_ = strip(_WHERE_)!!" &&fmtnewvar&i ='"!!tranwrd(strip(&&fmtnewvar&i), "'", "''")!!"'";
16069 +                       _WHEREDESC_ = strip(_WHEREDESC_)!!" &VarName = "!!strip(&&fmtnewvar&i);
16070 +              %end;
16071 +              %if &i < &numgroupvar %then %do;
16072 +                  _WHERE_ = strip(_WHERE_)!!' and ';
16073 +                  _WHEREDESC_ = strip(_WHEREDESC_)!!' and ';
16074 +              %end;
16075 +          %end;
16076 +          %if "&EM_PROPERTY_MODE" = "CROSSVALIDATION"  %then %do;
16077 +              _WHERE_ = '^('!!strip(_WHERE_)!!')';
16078 +              _WHEREDESC_ = '^('!!strip(_WHEREDESC_)!!')';
16079 +              FREQUENCY = &_totalobs_ - FREQUENCY;
16080 +          %end;
16081 +      run;
16083 +      %if (&EM_PROPERTY_TARGETGROUP= Y and &EM_NUM_TARGET>1) %then %do;
16084 +          data target;
16085 +             length _TARGET_ $32;
16086 +              %do i=1 %to &em_num_target;
16087 +                  %let varname = %scan(%em_target, &i, %str( ));
16088 +                  _TARGET_ = "&varname";
16089 +                  output;
16090 +              %end;
16091 +          run;
16092 +          proc sql;
16093 +             create table &em_user_groupinfo as select * from  work.temp, work.target;
16094 +          quit;
16095 +          data &em_user_groupinfo;
16096 +             set &em_user_groupinfo;
16097 +             _LOOP_=_N_;
16098 +          run;
16099 +      %end;
16101 +      data &em_data_eminfo;
16102 +         length TARGET KEY $32  DATA $43;
16103 +         TARGET=''; KEY='GROUPINFO'; DATA="&em_nodeid"; output;
16104 +      run;
16106 +      data &em_user_emloopinfo;
16107 +         set &em_user_groupinfo;
16108 +         %if (&EM_PROPERTY_TARGETGROUP= Y and &EM_NUM_TARGET>1) %then %do;
16109 +             label _LOOP_      = "%sysfunc(sasmsg(sashelp.dmine, rpt_groupIndex_vlabel, NOQUOTE))"
16110 +                   _TARGET_    = "%sysfunc(sasmsg(sashelp.dmine, rpt_target_vlabel,     NOQUOTE))"
16111 +                   _WHEREDESC_ = "%sysfunc(sasmsg(sashelp.dmine, rpt_where_vlabel,      NOQUOTE))"
16112 +                   FREQUENCY   = "%sysfunc(sasmsg(sashelp.dmine, rpt_count_vlabel,      NOQUOTE))";
16113 +             keep _LOOP_ _TARGET_ _WHEREDESC_ FREQUENCY;
16114 +         %end;
16115 +         %else %do;
16116 +             label _LOOP_      = "%sysfunc(sasmsg(sashelp.dmine, rpt_groupIndex_vlabel, NOQUOTE))"
16117 +                   _TARGET_    = "%sysfunc(sasmsg(sashelp.dmine, rpt_target_vlabel,     NOQUOTE))"
16118 +                   _WHEREDESC_ = "%sysfunc(sasmsg(sashelp.dmine, rpt_where_vlabel,      NOQUOTE))"
16119 +                   FREQUENCY   = "%sysfunc(sasmsg(sashelp.dmine, rpt_count_vlabel,      NOQUOTE))";
16120 +             keep _LOOP_ _WHEREDESC_ FREQUENCY;
16121 +         %end;
16122 +      run;
16124 +      proc datasets lib=work nolist;
16125 +          delete temp loopinfo
16126 +          %if (&EM_PROPERTY_TARGETGROUP= Y and &EM_NUM_TARGET>1) %then %do;
16127 +              target
16128 +          %end;
16129 +          ;
16130 +      run;
16132 +   %end;
16134 +   filename gpDelta "&EM_FILE_CDELTA_TRAIN";
16135 +   %if (&EM_PROPERTY_TARGETGROUP= Y and &EM_NUM_TARGET>1) %then %do;
16136 +       data _null_;
16137 +          length quotedTarget $32;
16138 +          set &em_user_groupinfo;
16139 +          file gpdelta;
16140 +          if _N_ = &emloopnumber then do;
16141 +             quotedTarget = "'"!!strip(_TARGET_)!!"'";
16142 +             put "if ROLE = 'TARGET' and NAME ne " quotedTarget " then delete;";
16144 +             %do i=1 %to &numgroupvar;
16145 +                 %let gvar = %upcase(%scan(&groupvar, &i, %str( )));
16146 +                 put "if upcase(NAME)= '&gvar' then delete;";
16147 +             %end;
16148 +          end;
16149 +       run;
16150 +   %end;
16151 +   %else %do;
16152 +       data _null_;
16153 +          set &em_user_groupinfo;
16154 +          file gpdelta;
16155 +          if _N_ = &emloopnumber then do;
16156 +             %do i=1 %to &numgroupvar;
16157 +              %let gvar = %upcase(%scan(&groupvar, &i, %str( )));
16158 +                 put "if upcase(NAME)= '&gvar' then delete;";
16159 +             %end;
16160 +          end;
16161 +       run;
16162 +   %end;
16163 +   filename gpdelta;
16165 +   %let nobs=0;
16166 +   %let dsid = %sysfunc(open(&em_user_groupinfo));
16167 +   %if &dsid>0 %then %do;
16168 +       %let nobs = %sysfunc(attrn(&dsid, NOBS));
16169 +       %let dsid = %sysfunc(close(&dsid));
16170 +   %end;
16172 +   %if ^&nobs  %then
16173 +       %let emloopmode =;
16174 +   %else
16175 +       %if &nobs< &emloopnumber %then
16176 +           %let emloopmode = POSTLOOP;
16177 +        %else
16178 +        %let emloopmode = LOOP;
16179 +%mend gp_group;
16181 +%macro gp_Index;
16183 +   %em_getName(key=INDEXINFO,  type=DATA);
16184 +   %em_getName(key=EMLOOPINFO, type=DATA);
16186 +   %if &emloopnumber=1 or ^%sysfunc(exist(&em_user_indexinfo)) %then %do;
16187 +       data &em_user_indexinfo;
16188 +          length _LOOP_ 8 _TARGET_ $32 _WHERE_ $8;
16189 +          _LOOP_     =&EM_PROPERTY_INDEXCOUNT;
16190 +          _TARGET_   ="";
16191 +          _WHERE_    ="";
16192 +          output;
16193 +       run;
16194 +       data &em_data_eminfo;
16195 +          length TARGET KEY $32  DATA $43;
16196 +          TARGET=''; KEY='INDEXINFO'; DATA="&em_nodeid"; output;
16197 +       run;
16199 +       data &em_user_emloopinfo;
16200 +          length MODE $8;
16201 +          label MODE     = "%sysfunc(sasmsg(sashelp.dmine, rpt_mode_vlabel,       NOQUOTE))"
16202 +                _LOOP_   = "%sysfunc(sasmsg(sashelp.dmine, rpt_groupIndex_vlabel, NOQUOTE))"
16203 +          set &em_user_indexinfo(keep=_LOOP_ );
16204 +          MODE='Index';
16205 +       run;
16206 +   %end;
16208 +%mend gp_index;
16210 +%macro gp_Target;
16211 +   %let TargetName = %scan(%em_target, &emloopnumber);
16212 +   %if "&targetName" eq "" %then %do;
16213 +       %goto doendgpt;
16214 +   %end;
16216 +   %if %index(&EM_DEBUG, _ALL_) %then %do;
16217 +       %put GP_TARGET;
16218 +       %put &emLoopnumber &TargetName;
16219 +   %end;
16221 +   filename gpDelta "&EM_FILE_CDELTA_TRAIN";
16222 +   data _null_;
16223 +      file gpdelta;
16224 +      put "if ROLE = 'TARGET' and NAME ne '&targetName' then delete;";
16225 +   run;
16226 +   filename gpdelta;
16228 +   %if &emloopnumber eq 1 %then %do;
16229 +       %em_getName(key=TARGETINFO,  type=DATA);
16230 +       %em_getName(key=EMLOOPINFO,  type=DATA);
16232 +       data &em_user_targetinfo;
16233 +          length _LOOP_ 8 _TARGET_ $32 _WHERE_ $2000;
16234 +          retain _LOOP_;
16235 +          _WHERE_ = '';
16236 +          %do i=1 %to &em_num_target;
16237 +              %let varname = %scan(%em_target, &i, ' ');
16238 +              _TARGET_ = "&varname";
16239 +              _LOOP_+1;
16240 +              output;
16241 +           %end;
16242 +       run;
16243 +       data &em_user_emloopinfo;
16244 +         set &em_user_targetinfo(keep=_LOOP_ _TARGET_);
16245 +         label _LOOP_      = "%sysfunc(sasmsg(sashelp.dmine, rpt_groupIndex_vlabel, NOQUOTE))"
16246 +               _TARGET_    = "%sysfunc(sasmsg(sashelp.dmine, rpt_target_vlabel,     NOQUOTE))";
16247 +      run;
16249 +       data &em_data_eminfo;
16250 +          length TARGET KEY $32  DATA $43;
16251 +          TARGET=''; KEY='TARGETINFO'; DATA="&em_nodeid"; output;
16252 +       run;
16253 +   %end;
16255 +   %doendgpt:
16256 +%mend gp_target;
16258 +%macro train;
16259 +   %if %index(&EM_DEBUG, _ALL_) %then %do;
16260 +       %put LOOP MODE:   &emloopmode ;
16261 +       %put LOOP NUMBER: &emloopnumber;
16262 +   %end;
16264 +   %if &emloopmode eq 'POSTLOOP' %then %goto doendmain;
16266 +   %if ("&EM_PROPERTY_MODE" = "BOOSTING") or ("&EM_PROPERTY_MODE" = "BAGGING") %then %do;
16268 +       %let hpdmFlag = 0;
16269 +       %if %symexist(em_import_DATA_eminfo) %then %do;
16270 +            data _null_;
16271 +               set &em_import_DATA_eminfo;
16272 +               where KEY = "HPDMSAMPLE";
16273 +               call symput('hpdmFlag', '1');
16274 +            run;
16275 +       %end;
16276 +       %if &hpdmFlag %then %do;
16277 +           %let emexceptionstring = exception.server.EMTOOL.HPNOBOOSTBAG;
16278 +           %goto doendmain;
16279 +       %end;
16280 +   %end;
16282 +   %em_getname(key=EMLOOPINFO,  type=DATA);
16283 +   %em_getname(key=GROUPINFO,  type=DATA);
16284 +   %em_getname(key=BOOSTINFO,  type=DATA);
16285 +   %em_getname(key=BAGINFO,    type=DATA);
16286 +   %em_getname(key=TARGETINFO, type=DATA);
16287 +   %em_getname(key=INDEXINFO,  type=DATA);
16288 +   %em_getname(key=EMINFO,  type=DATA);
16290 +   %if "&emloopnumber" = "1" %then %do;
16291 +       %em_getname(key=PRECODE, type=FILE, extension=sas);
16292 +       filename _pregrp "&em_user_precode";
16293 +       data _null_;
16294 +         rc = fdelete('_pregrp');
16295 +       run;
16296 +       filename _pregrp;
16298 +       %let members =;
16299 +       %if %sysfunc(exist(&em_user_emloopinfo)) %then %let members = &members %scan(&EM_USER_EMLOOPINFO,   2, .);
16300 +       %if %sysfunc(exist(&em_user_groupinfo))  %then %let members = &members %scan(&EM_USER_GROUPINFO,  2, .);
16301 +       %if %sysfunc(exist(&em_user_targetinfo)) %then %let members = &members %scan(&EM_USER_TARGETINFO, 2, .);
16302 +       %if %sysfunc(exist(&em_user_boostinfo))  %then %let members = &members %scan(&EM_USER_BOOSTINFO,  2, .);
16303 +       %if %sysfunc(exist(&em_user_baginfo))    %then %let members = &members %scan(&EM_USER_BAGINFO,    2, .);
16304 +       %if %sysfunc(exist(&em_user_indexinfo))  %then %let members = &members %scan(&EM_USER_INDEXINFO,  2, .);
16305 +       %if %sysfunc(exist(&em_user_eminfo))     %then %let members = &members %scan(&EM_USER_EMINFO,  2, .);
16307 +       %if "&members" ne "" %then %do;
16308 +           proc datasets lib=&em_lib nolist;
16309 +              delete &members;
16310 +           run;
16311 +      %end;
16312 +   %end;
16314 +   %let emboostvar    =;
16315 +   %let embagvar      =;
16316 +   %let emgroupvar    =;
16317 +   %let emnumgroupvar =;
16318 +   %getVariables;
16320 +   %if &EM_PROPERTY_MODE = TARGET %then %do;
16321 +       %if &EM_NUM_TARGET>=1 %then %do;
16322 +           %gp_target;
16323 +           %if &emLoopNumber <= &EM_NUM_TARGET %then
16324 +               %let emloopmode= LOOP;
16325 +           %else
16326 +              %let emloopmode = POSTLOOP;
16327 +       %end;
16328 +       %else %do;
16329 +           %let emloopmode = ;
16330 +           %let emexceptionstring = exception.server.METADATA.USE1TARGET;
16331 +           %goto doendmain;
16332 +       %end;
16333 +   %end;
16334 +   %else
16335 +   %if &EM_PROPERTY_MODE = NOGROUP %then %do;
16336 +       %let emloopmode=;
16337 +       %if %sysfunc(exist(&em_data_eminfo)) %then %do;
16338 +           proc delete data=&em_data_eminfo;
16339 +           run;
16340 +       %end;
16341 +   %end;
16342 +   %else
16343 +   %if &EM_PROPERTY_MODE = INDEX %then %do;
16344 +       %gp_index;
16345 +       %if  &emLoopNumber <= &EM_PROPERTY_INDEXCOUNT %then
16346 +           %let emloopmode= LOOP;
16347 +       %else
16348 +           %let emloopmode= POSTLOOP;
16349 +   %end;
16350 +   %else
16351 +   %if "&EM_PROPERTY_MODE" = "STRATIFY" or "&EM_PROPERTY_MODE" = "CROSSVALIDATION"  %then %do;
16352 +       %if "&emloopnumber"="1" and "&emgroupvar" eq "" %then %do;
16353 +           %let emloopmode = ;
16354 +           %let emexceptionstring = exception.server.METADATA.USE1GROUPVAR;
16355 +           %goto doendmain;
16356 +       %end;
16358 +       %gp_group(numgroupvar=&emnumgroupvar, groupvar=&emgroupvar);
16359 +   %end;
16360 +   %else
16361 +   %if &EM_PROPERTY_MODE = BOOSTING %then %do;
16362 +       %if "&emboostvar" eq "" %then %do;
16363 +           %let emloopmode = ;
16364 +           %let emexceptionstring = exception.server.METADATA.USE1BOOSTVAR;
16365 +           %goto doendmain;
16366 +       %end;
16368 +       %if  &emLoopNumber <= &EM_PROPERTY_INDEXCOUNT %then %do;
16369 +           %let emloopmode= LOOP;
16370 +           %gp_boost(targetname=&emboostvar);
16372 +       %end;
16373 +       %else
16374 +           %let emloopmode= POSTLOOP;
16375 +   %end;
16376 +   %else
16377 +   %if &EM_PROPERTY_MODE = BAGGING %then %do;
16378 +       %if "&embagvar" eq "" %then %do;
16379 +           %let emloopmode = ;
16380 +           %let emexceptionstring = exception.server.METADATA.USE1BAGVAR;
16381 +           %goto doendmain;
16382 +       %end;
16384 +        %if  &emLoopNumber <= &EM_PROPERTY_INDEXCOUNT %then %do;
16385 +           %let emloopmode= LOOP;
16386 +           %gp_bag(targetname=&embagvar);
16387 +       %end;
16388 +       %else
16389 +           %let emloopmode= POSTLOOP;
16390 +   %end;
16393 +   %if &emloopmode eq "" %then %do;
16394 +       %let lib    = %scan(&em_data_eminfo, 1, .);
16395 +       %let member = %scan(&em_data_eminfo, 2, .);
16396 +       proc datasets lib=&lib nolist;
16397 +          delete &member;
16398 +       run;
16399 +   %end;
16401 +   %doendmain:
16403 +   %if %index(&EM_DEBUG, _ALL_) %then %do;
16404 +       %put NEW LOOP MODE: &emloopmode;
16405 +   %end;
16407 +   %if "&emloopmode" eq "POSTLOOP" %then %do;
16408 +       filename temp catalog 'sashelp.emutil.gp_postloop.source';
16409 +       %include temp;
16410 +       filename temp;
16412 +       %postloop;
16413 +   %end;
16415 +%mend train;
NOTE: %INCLUDE (level 1) ending.
NOTE: Fileref TEMP has been deassigned.
ERROR: File WORK.METASET30JQ462.DATA does not exist.
 
NOTE: The SAS System stopped processing this step because of errors.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      user cpu time       0.01 seconds
      system cpu time     0.00 seconds
      memory              23242.96k
      OS Memory           32624.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  0
      Page Faults                       0
      Page Reclaims                     21
      Page Swaps                        0
      Voluntary Context Switches        0
      Involuntary Context Switches      0
      Block Input Operations            0
      Block Output Operations           0
 
 
 
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      user cpu time       0.00 seconds
      system cpu time     0.00 seconds
      memory              23242.96k
      OS Memory           32624.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  0
      Page Faults                       0
      Page Reclaims                     21
      Page Swaps                        0
      Voluntary Context Switches        0
      Involuntary Context Switches      0
      Block Input Operations            0
      Block Output Operations           0
 
 
NOTE: Fileref _PREGRP has been deassigned.
 
NOTE: Deleting EMWS3.GRP2_BOOSTINFO (memtype=DATA).
NOTE: Deleting EMWS3.GRP2_EMINFO (memtype=DATA).
 
NOTE: PROCEDURE DATASETS used (Total process time):
      real time           0.01 seconds
      user cpu time       0.00 seconds
      system cpu time     0.00 seconds
      memory              23242.96k
      OS Memory           32624.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  0
      Page Faults                       0
      Page Reclaims                     125
      Page Swaps                        0
      Voluntary Context Switches        35
      Involuntary Context Switches      1
      Block Input Operations            32
      Block Output Operations           8
 
 
 
NOTE: There were 1 observations read from the data set EMWS3.GRP2_VARIABLESET.
      WHERE (role='TARGET') and (level not = 'INTERVAL') and grouprole in ('BOOST', 'DEFAULT');
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      user cpu time       0.00 seconds
      system cpu time     0.00 seconds
      memory              23487.96k
      OS Memory           32876.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  0
      Page Faults                       0
      Page Reclaims                     97
      Page Swaps                        0
      Voluntary Context Switches        2
      Involuntary Context Switches      0
      Block Input Operations            0
      Block Output Operations           0
 
 
NOTE: %INCLUDE (level 1) file TEMP is file SASHELP.EMUTIL.GP_BOOSTMACROS.SOURCE.
16417 +%macro embag(_tra=, _train=, _seed=, _freq=, _sumfreq=, _size=, _loop=);
16418 +  data &_tra;
16419 +    set &_train;
16420 +
16421 +    retain seed nremain ntrials ptrials -1;
16422 +    drop   seed nremain ntrials ptrials;
16423 +
16424 +    * If a frequency variable exists, do not export it                 ;
16425 +    %if %bquote(&_freq) ^=  %then %do;
16426 +        drop &_freq ;
16427 +        if &_freq eq . then return ;
16428 +        if &_freq <= 0 then return ;
16429 +    %end;
16430 +
16431 +    if ntrials lt 0 then do;     * Initialization                      ;
16432 +        seed = &_seed + &_loop;  * Seed depends on loop index          ;
16433 +        ntrials = round(&_size); * Number of observations to select    ;
16434 +        nremain = &_sumfreq;     * Obs to choose from = sum of freq var;
16435 +    end;
16436 +
16437 +    * The remaining observations are equally likely to be chosen       ;
16438 +    ptrials  = 1 / nremain %str(;);
16439 +
16440 +    * Incorporate any frequency variable into probability of selection ;
16441 +    %if %bquote(&_freq) ^=  %then ptrials   = &_freq * ptrials %str(;);
16442 +
16443 +    * Randomly set _resamp_ = number of times to output this obs       ;
16444 +    if ptrials < 1 then call ranbin( seed, ntrials, ptrials, _resamp_);
16445 +    else                _resamp_ = ntrials;
16446 +
16447 +    * Output training observation                                      ;
16448 +    if _resamp_ > 0 then output &_tra;
16449 +
16450 +    * Update number of observations still needed to output             ;
16451 +    ntrials = ntrials - _resamp_;
16452 +
16453 +    * If output enough observatoins, stop.                             ;
16454 +    if ntrials <= 0 then stop;
16455 +
16456 +    * Update number of observations remaining to choose from           ;
16457 +    %if %bquote(&_freq) ^=  %then nremain   = nremain  - &_freq %str(;);
16458 +    %else                         nremain   = nremain  -      1 %str(;);
16459 +
16460 +  run;
16461 +
16462 +%mend embag;
16463 +
16464 +%macro emboost0( _train=, _dmboods=, _target=, _freq=);
16465 +   %put _TRAIN: &_train _DMBOODS: &_DMBOODS _TARGET:&_TARGET _FREQ &_FREQ;
16466 +   %global _sumfreq _dmboodiv;
16467 +
16468 +   * MAINTAIN OBSERVATION WEIGHTS ETC IN the _DMBOODS DATA SET ...;
16469 +
16470 +   data &_dmboods;
16471 +      set &_train end = the_end;
16472 +      retain sumfreq 0;
16473 +      keep _boostwt _nmissed &_freq;
16474 +
16475 +      * WILL COUNT NUMBER OF MODELS THAT MISCLASSIFY THE OBSERVATION ;
16476 +      _nmissed    = 0;
16477 +
16478 +      * INITIALIZE OBSERVATION WEIGHT TO OBS FREQUENCY ;
16479 +      %if %bquote(&_freq) NE %then %do ;
16480 +          _boostwt = &_freq ;
16481 +      %end ;
16482 +      %else %do ;
16483 +          _boostwt = 1;
16484 +      %end ;
16485 +
16486 +      * ADD FREQUENCY OF OBSERVATIONS IN TRAINING DATA ;
16487 +      * OUTPUT OBSERVATION ;
16488 +      if _boostwt gt 0 then do ;
16489 +         sumfreq  = sumfreq + _boostwt;
16490 +         output ;
16491 +      end ;
16492 +
16493 +      * AFTER ALL OBSERVATIONS ARE READ, SET GLOBAL MACROS ;
16494 +      if the_end ne 0 then do;
16495 +         call symput('_sumfreq', trim(left(put(sumfreq,best12.))));
16496 +         call symput('_dmboodiv', '1');
16497 +      end;
16498 +   run;
16499 +%mend emboost0;
16500 +
16501 +%macro emboost2(_freq=, _dmboods=, _dmensds=, _dmboosf=, _sumfreq=);
16502 +%put _FREQ: &_freq _DMBOODS: &_DMBOODS _DMNENSDS: &_dmensds _DMBOOSF: &_dmboosf  _SUMFREQ: &_sumfreq;
16503 +
16504 +   data &_dmboods;
16505 +      set  &_dmboods end=the_end;
16506 +      set  &_dmensds;
16507 +      keep _boostwt _nmissed &_freq;
16508 +      retain sum 0;
16509 +
16510 +      * If a frequency variable exists, check for positive value    ;
16511 +      %if %bquote(&_freq) ^=  %then %do;
16512 +          if &_freq eq . then return ;
16513 +          if &_freq <= 0 then return ;
16514 +      %end;
16515 +
16516 +      * IF OBSERVATION IS MISCLASSIFIED, INCREMENT COUNT    ;
16517 +      if f_&_dmboosf ne i_&_dmboosf then _nmissed = _nmissed +1;
16518 +
16519 +      * Base observation weight on how poorly models have fit it.    ;
16520 +      * Use formula suggested in Arching Classifiers by Leo Breiman. ;
16521 +      _boostwt =  1 + _nmissed**4;
16522 +
16523 +      * MULTIPLY WEIGHT BY FREQUENCY VARIABLE, IF ONE EXISTS        ;
16524 +      %if %bquote(&_freq) ^=  %then
16525 +          _boostwt = _boostwt * &_freq %str(;);
16526 +
16527 +      sum = sum + _boostwt;
16528 +      output;
16529 +
16530 +      if the_end ne 0 then do;
16531 +         sum = sum/ &_sumfreq;
16532 +         call symput('_dmboodiv', strip(put(sum,best12.)));
16533 +      end;
16534 +   run;
16535 +%mend emboost2;
16536 +
16537 +%macro emboost1(_tra=, _train=, _freq=, _dmboods=, _dmboodiv=);
16538 +
16539 +   data &_tra;
16540 +      set &_train;
16541 +      set &_dmboods;
16542 +      drop _boostwt _nmissed &_freq;
16543 +
16544 +      * If a frequency variable exists, exlude obs with non-positive  ;
16545 +      %if %bquote(&_freq) ^=  %then %do;
16546 +          if &_freq eq . then return ;
16547 +          if &_freq <= 0 then return ;
16548 +      %end;
16549 +
16550 +      _resamp_ = _boostwt / &_dmboodiv;
16551 +      output;
16552 +   run;
16553 +
16554 +%mend emboost1;
16555 +
NOTE: %INCLUDE (level 1) ending.
NOTE: Fileref TEMP has been deassigned.
 
NOTE: The file GPDELTA is:
      Filename=/home/mercep830/DATA 640/Workspaces/EMWS3/Grp2/CDELTA_TRAIN.sas,
      Owner Name=mercep830,Group Name=oda,
      Access Permission=-rw-r--r--,
      Last Modified=01Aug2017:16:40:40
 
NOTE: 4 records were written to the file GPDELTA.
      The minimum record length was 4.
      The maximum record length was 61.
NOTE: DATA statement used (Total process time):
      real time           0.01 seconds
      user cpu time       0.01 seconds
      system cpu time     0.00 seconds
      memory              23487.96k
      OS Memory           33132.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  0
      Page Faults                       0
      Page Reclaims                     28
      Page Swaps                        0
      Voluntary Context Switches        4
      Involuntary Context Switches      0
      Block Input Operations            0
      Block Output Operations           8
 
 
NOTE: Fileref GPDELTA has been deassigned.
 
NOTE: Deleting EMWS3.GRP2_BOOSTWEIGHT (memtype=DATA).
_TRAIN: EMWS3.FIMPORT_train _DMBOODS: EMWS3.Grp2_BOOSTWEIGHT _TARGET:Churn _FREQ
 
NOTE: PROCEDURE DATASETS used (Total process time):
      real time           0.00 seconds
      user cpu time       0.00 seconds
      system cpu time     0.01 seconds
      memory              23487.96k
      OS Memory           33132.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  0
      Page Faults                       0
      Page Reclaims                     47
      Page Swaps                        0
      Voluntary Context Switches        21
      Involuntary Context Switches      1
      Block Input Operations            32
      Block Output Operations           8
 
 
 
NOTE: View EMWS3.FIMPORT_TRAIN.VIEW used (Total process time):
      real time           0.02 seconds
      user cpu time       0.01 seconds
      system cpu time     0.01 seconds
      memory              89781.53k
      OS Memory           98676.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  5
      Page Faults                       0
      Page Reclaims                     223
      Page Swaps                        0
      Voluntary Context Switches        18
      Involuntary Context Switches      0
      Block Input Operations            0
      Block Output Operations           8
 
NOTE: There were 2084 observations read from the data set EMWS3.FIMPORT_DATA.
NOTE: There were 2084 observations read from the data set EMWS3.FIMPORT_TRAIN.
NOTE: The data set EMWS3.GRP2_BOOSTWEIGHT has 2084 observations and 2 variables.
NOTE: DATA statement used (Total process time):
      real time           0.04 seconds
      user cpu time       0.01 seconds
      system cpu time     0.01 seconds
      memory              89781.53k
      OS Memory           98676.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  6
      Page Faults                       0
      Page Reclaims                     353
      Page Swaps                        0
      Voluntary Context Switches        30
      Involuntary Context Switches      4
      Block Input Operations            0
      Block Output Operations           264
 
 
 
NOTE: The data set EMWS3.GRP2_BOOSTINFO has 1 observations and 4 variables.
NOTE: DATA statement used (Total process time):
      real time           0.02 seconds
      user cpu time       0.00 seconds
      system cpu time     0.00 seconds
      memory              89781.53k
      OS Memory           98676.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  0
      Page Faults                       0
      Page Reclaims                     74
      Page Swaps                        0
      Voluntary Context Switches        20
      Involuntary Context Switches      4
      Block Input Operations            0
      Block Output Operations           264
 
 
 
NOTE: There were 1 observations read from the data set EMWS3.GRP2_BOOSTINFO.
NOTE: The data set EMWS3.GRP2_EMLOOPINFO has 1 observations and 3 variables.
NOTE: DATA statement used (Total process time):
      real time           0.01 seconds
      user cpu time       0.01 seconds
      system cpu time     0.00 seconds
      memory              89781.53k
      OS Memory           98676.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  0
      Page Faults                       0
      Page Reclaims                     458
      Page Swaps                        0
      Voluntary Context Switches        31
      Involuntary Context Switches      12
      Block Input Operations            288
      Block Output Operations           264
 
 
 
NOTE: The data set EMWS3.GRP2_EMINFO has 1 observations and 3 variables.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      user cpu time       0.00 seconds
      system cpu time     0.00 seconds
      memory              89781.53k
      OS Memory           98676.00k
      Timestamp           08/01/2017 04:40:40 PM
      Step Count                        1  Switch Count  0
      Page Faults                       0
      Page Reclaims                     74
      Page Swaps                        0
      Voluntary Context Switches        20
      Involuntary Context Switches      6
      Block Input Operations            0
      Block Output Operations           272
 
 
16556  *------------------------------------------------------------*;
16557  * End TRAIN: Grp2;
16558  *------------------------------------------------------------*;
16559
 
*------------------------------------------------------------*
*
* ERROR: Run time error was encountered.  The system error returned was 1012.
* Please report unresolved problems to Technical Support.
*
*------------------------------------------------------------*
1 ACCEPTED SOLUTION

Accepted Solutions
MerceraS
Fluorite | Level 6

Hi, Doug.

I actually found the answer in another problem note, 55675, which describes in great detail what the problem is and how to avoid it. The use of the File Import node at the beginning of a flow ending with Start Groups and End Groups nodes set to Bagging or Boosting is what causes the vague error message. To avoid it, you need to convert the imported file into a SAS dataset using one of several different methods or use a pre-loaded dataset in the SAS Enterprise Miner library. Please see Problem Note 55675 for more information so you can help others who experience this error. I lost a lot of time with my homework trying to figure this out when the answer was already solved.

View solution in original post

5 REPLIES 5
WendyCzika
SAS Employee

Please contact SAS Technical Support for this issue, with information like what your exact flow is: https://support.sas.com/en/technical-support.html

MerceraS
Fluorite | Level 6
Hi, Wendy. I reached out to Technical Support two days ago and they suggested that I post here for assistance. They said they only help with start up issues like setting up one's account and accessing SAS products like Enterprise Miner. They could not offer any more assistance, unfortunately, which is why I'm posting here.
DougWielenga
SAS Employee

Enterprise Miner general usage support for SAS OnDemand for Academics (or SODA) is handled through the SAS Communities.  If you have this type of license, you would have been instructed to go to https://communities.sas.com/community/sas-analytics-u to seek assistance since many other people using the software likely would encounter the same question.   

 

In your case, the error is a generic one that usually can be isolated to the data or variables.  In your case, you are performing group processing which means that if the modeling loop failed during any of the loops, the flow will fail.  This makes it particularly hard to diagnose since it is impossible to isolate the details from the failed loop; however, there are things we can look at to investigate.

 

In your case, share the following information:

1 - The results of running the CONTENTS procedure against the Input Data Source

2 - A model package created from the End Groups node

 

To accomplish item 1 above, proceed as follows:

   a. Add and connect a SAS Code node following the Input Data Source node (note: the SAS Code node can be found among the nodes on the Utility tab)

   b. Click on the ... to the right of Code Editor in the property sheet for the SAS Code node you added

   c.  Add the following code to the Training code section

 

/*** BEGIN SAS CODE ***/

 

proc contents data = &EM_IMPORT_DATA;

run;

 

/*** END SAS CODE ***/

    d.  Close the node saving the changes when prompted.  

    e.  Run the node from the SAS Code node.

    f.   View the Results of running the CONTENTS procedure and attach (using File --> Save As) or cut and paste the results into your reply.

 

To accomplish item 2 above, follow the instructions to complete a model package from the End Groups node in SAS Note 46764 - Creating a model package (.SPK file) in SAS® Enterprise Miner(tm) which is available at 

 

http://support.sas.com/kb/46/764.html

 

Please note that the model package will be created even if an error message appears after completing the steps.  The model package will help us investigate further.

 

I look forward to your response!

 

Cordially,

Doug

 

 

MerceraS
Fluorite | Level 6

Hi, Doug.

I actually found the answer in another problem note, 55675, which describes in great detail what the problem is and how to avoid it. The use of the File Import node at the beginning of a flow ending with Start Groups and End Groups nodes set to Bagging or Boosting is what causes the vague error message. To avoid it, you need to convert the imported file into a SAS dataset using one of several different methods or use a pre-loaded dataset in the SAS Enterprise Miner library. Please see Problem Note 55675 for more information so you can help others who experience this error. I lost a lot of time with my homework trying to figure this out when the answer was already solved.

DougWielenga
SAS Employee

Please see Problem Note 55675 for more information so you can help others who experience this error. I lost a lot of time with my homework trying to figure this out when the answer was already solved.

 

I'm glad that you found your solution, and am sorry it took as long as it did to identify it.   As I mentioned in my previous response, the error message you received was a generic one which has been reported in many different nodes for the reasons I mentioned.  I requested that you post a model package because that would have provided the context (the use of the File Import node, for instance) which would have helped isolate this issue to the one identified in the SAS note you found.  Without the model package, there is little information to eliminate the many other issues which might have led to the same error message you reported.    

 

In the future, I would encourage you to post a model package with your questions.  It is usually the first thing requested by SAS Technical Support for SAS Enterprise Miner questions because it provides a great deal of information including the version number, operating system, sequence of nodes and settings, logs, and node results but it does not include your data.  This provides the best chance of getting to the correct issue quickly.   

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to choose a machine learning algorithm

Use this tutorial as a handy guide to weigh the pros and cons of these commonly used machine learning algorithms.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 5 replies
  • 12454 views
  • 4 likes
  • 3 in conversation