ledfoot Tracker

Re: SAS Enterprise Miner | SASENVIROMENTSURL

ledfoot — Fri, 21 Sep 2018 19:05:56 GMT

I'm not 100% sure what your goal is but I'll offer this.

We have 2 different Windows machines with EM running. On the development machine I found C:\SAS\Config\Lev1\Web\WebServer\htdocs\sas\sas-environment.xml.

I made a backup copy of the file before modified it.

I copied the default <environment> section and pasted it below the original.
I changed the first <environment> to name="Dev" and <desc> to Development SAS Environment.

In the new/copied <environment> I changed name="Prod", <desc> to Production SAS Environment and pointed the <service-registry> to our production machine.

When I logged in to EM on my development machine it gave me a choice of environments.

In the following environments.xml I redacted MACHINENAME1 and MACHINENAME2:

<?xml version="1.0" encoding="UTF-8"?>
<environments xmlns="http://www.sas.com/xml/schema/sas-environments-9.4"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.sas.com/xml/schema/sas-environments-9.4
http://www.sas.com/xml/schema/sas-environments-9.4/sas-environments-9.4.xsd"
version="2.0">
<environment name="DEV" default="true" platform-version="9.4">
<desc>Development SAS Environment</desc>
<service-registry>
http://MACHINENAME1:80/SASWIPClientAccess/remote/ServiceRegistry
</service-registry>
<service-registry interface-type="soap">
http://MACHINENAME1:80/SASWIPSoapServices/services/ServiceRegistry
</service-registry>
<service-registry interface-type="rest">http://MACHINENAME1:80/SASWIPClientAccess/rest/</service-registry>
</environment>
<environment name="PROD" default="false" platform-version="9.4">
<desc>Production SAS Environment</desc>
<service-registry>
http://MACHINENAME2:80/SASWIPClientAccess/remote/ServiceRegistry
</service-registry>
<service-registry interface-type="soap">
http://MACHINENAME2:80/SASWIPSoapServices/services/ServiceRegistry
</service-registry>
<service-registry interface-type="rest">http:MACHINENAME2:80/SASWIPClientAccess/rest/</service-registry>
</environment>
</environments>

I hope this helps.

Re: Open/convert .dmp

ledfoot — Fri, 07 Sep 2018 14:39:11 GMT

By default EM does not produce files with a 'dmp' extension. There are several types of files EM can produce that the User can name. Without more to go on it is difficult to guess what you have.

Where did the file come from?

What do you hope to do with the file?

What have you tried to open it with?

Re: Difference between enterprise miner client and EM workstation

ledfoot — Thu, 26 Jul 2018 16:23:36 GMT

To be clear let’s establish a common lexicon. You can install EM in 2 basic forms.

These are described in the installation guide as "Client/Server" and "Workstation".

Both provide the graphical user interface referred to as the "Client".

So if you were running both it would not be immediately apparent which was which. The EM interface (Client) is identical on both Client/Server and Workstation installs unless you dig into the details.

Client/Server installs are the whole "enterprise" level, massively multiuser, model training, scoring, deploying and sharing, Swiss Army knife that is the envy of the data mining world.

(Can you tell I like it?)

If you have the hardware you can install the EM server on one big machine and install just the Client on as many smaller machine as you have licenses for and they can all share the big server where most of the work happens.

For customers with an appreciation of data mining in general but smaller user base and budget SAS provides the Workstation install.

Workstation only installs on Windows but it has a much smaller footprint on the machine than Client/Server.

Re: Generating JAVA scoring code from SAS EM

ledfoot — Mon, 30 Apr 2018 16:47:19 GMT

The Score Code Export node does not export java code. It only exports SAS code.

To get the Score node to generate java code for a supported model, select the score node and in the properties under Score Code Generation set Java Score to Yes and run the Score node.

From that point the step by step instructions are in the document you linked.

Look under Scoring with Java Code and Save and Organize Java Code Component Files.

Re: SAS Enterprise Miner project conversion

ledfoot — Wed, 11 Apr 2018 12:32:40 GMT

Given the number of releases in between Enterprise Miner v5.3 and Enterprise Miner 14.1, you are probably best-served by creating a new project in the recent Enterprise Miner release, and building a new diagram instead of trying to convert something that old. But if you really want to try here are some suggestions and information.

Remember what you are attempting is not really supported. But lots of people have attempted this for reasons only they understand.

If your v14.1 image is on the same operating system as your v5.3 image you may be able to open the old project in 14.1 using Create new project with the name of the old project. That process is covered in some detail at...

http://support.sas.com/kb/32/904.html

and

https://communities.sas.com/t5/SAS-Data-Mining-and-Machine/Copy-Enterprise-Miner-Projects/m-p/99819#M828

However, since 5.3 was so long ago you probably need to migrate the old project from a 32 bit system to the the new 64 bit system. Information on this adventure is available at ...

http://support.sas.com/kb/32/905.html

Re: Enterprise Miner Question - Scoring Node

ledfoot — Mon, 13 Feb 2017 14:23:37 GMT

I looks like you are going to need Tech Support to figure out how to configure your system correctly. The errors indicate that the DTJ_CLASSPATH was not setup correctly when the system was installed.

Re: Enterprise Miner Question - Scoring Node

ledfoot — Mon, 13 Feb 2017 12:55:58 GMT

What operating system is the EM server on?

Re: Enterprise Miner Question - Scoring Node

ledfoot — Mon, 13 Feb 2017 12:52:55 GMT

Something is definately wrong. You should probably contact SAS Tech Support.

In the mean time can you share a pic of your EM flow and the log from the score node?

Re: Enterprise Miner Question - Scoring Node

ledfoot — Fri, 10 Feb 2017 19:08:37 GMT

It’s close but it looks like you might have skipped a step in the instructions.

In the Java Score Code window there should be 5 choices in the drop down box.

Scoring Function Metadata

User-defined Formats

Score Code

File Metadata

Java Class

The "Scoring Function Metadata" is an XML description of the models Input variables and Output variables. It is very handy for automating things like choosing what data to feed the model when you run it.

The "User-defined Formats" is Java code that is often just a stub unless your training data contains user defined formats.

The "Score Code" is Java code that invokes the model that is in the "Java Class".

The "File Metadata" is another XML file that describes the mime types of the other files.

The "Java Class" file is a binary java class file in which the actual model resides. One way to think of it is as "pre-compiled" java code.

The instruction are trying to say ...

Save the "Scoring Function Metadata" as JScore.xml.

Save the "User-defined Formats" as JscoreUserFormats.java.

Save the "Score Code" as Score.java.

Save the "Java Class" as DS.class.

Re: Enterprise Miner Question - Scoring Node

ledfoot — Fri, 10 Feb 2017 15:57:46 GMT

The EM Java scoring code is only available as a .class file. There is no .java file. The "Score Code" in the Results viewer is only a wrapper that calls the "Java Class" file (DS.class) that does the actual scoring. Note, since the DS.class file is not created using a java compiler it cannot be reliably decompiled.

Scoring Series Part 3: Enterprise Miner Optimized Score code

ledfoot — Tue, 06 Oct 2015 15:13:35 GMT

Why it’s important, how it’s created, and where it’s found

Why is optimized Score code important?

As early as the 1950s, businesses began to recognize the potential benefits of computers. From the demands of commercial enterprise rose the mainframe computer. And the mainframe computers in turn completely revolutionized how businesses operate. Computer development started to accelerate.

In 1962, one of the world's first supercomputers was officially commissioned at Manchester University. The Manchester University Atlas 1 was a joint development between the University of Manchester, Ferranti International PLC, and The Plessey Company PLC. It was considered to be the most powerful computer in the world at that time.

The Manchester Atlas 1 had 96 KB of core storage and 576 KB of virtual memory and could do a floating-point multiply in 4.97 microseconds (4.97e-6 FLOPS). High-level languages used to write programs for the Atlas included Algol 60, FORTRAN and COBOL.

At the time, great care was taken to carefully design and write programs to operate within the constraints of the system. The operating system reserved about 17 KB of the available 96 in core memory. Just try to imagine writing a program to run in a maximum 79KB. The complexity of the problems that were even considered was limited by the resources available to the program. With such constraints, programmers worried about every byte and often every instruction. By necessity, efficiency was the watchword of programming.

In the time between 1962 and now, the available hardware and software has progressed. We now have single systems offering terabytes of core memory and supporting over 120 GFLOPS or 1.2e+14 FLOPS. As the systems grew and the resources available to programmers increased, programs were able to do more and became more and more complex. The constant pressure to manage every byte and instruction gradually eroded. For a lot of practical reasons, the priority of program efficiency became secondary if not eliminated. After all, with seemingly unlimited and constantly increasing resources available, why worry about a few bytes or instructions? Over time, the consideration of program efficiency drew less and less attention. The rules changed.

If only the resources available to programs had expanded since 1962, programmers could easily have rested on their laurels. But snap to the age of BIG DATA! The world's capacity to store information has almost doubled every 3.3 years since the 1980. Some speculate that since 2012, 2.5 exabytes (2.5×1018) of data were created daily. The amount of data available to be processed has now turned back the clock to once again make efficiency of program design and implementation a critical issue for businesses and programmers. The rules have changed again.

Now with the ever-increasing amounts of data available to businesses and governments, even a small efficiency, multiplied by big data, becomes important - even critical.

How does the Score node accumulate and optimize the score code, and how do I save it in a package?

The process of optimizing the Enterprise Miner scoring source code is not applicable to all Enterprise Miner models but most do benefit. To avoid getting too far in to the realm of dragons and magicians, let’s characterize the process of Enterprise Miner (EM) score code accumulation by the Score node.

The Score node back traces the Enterprise Miner flow it is connected to, from end to beginning. Once at the beginning, the process picks up the score code produced by each node in the flow, in order. The Score node can then determine which variables are actually used in the final model. In the Score node results this code is displayed as “SAS Code”. By default the Score node property Optimized Code under Score Code Generation is set to “Yes”, which triggers the Score node to go through all the code it collected from the flow and omit any code that is not related to the variables used by the final model. This results in the code displayed as “Optimized SAS Code” in the Score node results. So in EM whenever possible both the optimized and un-optimized code are produced.

An EM model package doesn’t try to package both types of score code. If your situation requires a package with the non-optimized code, you will need to set the Score node property Optimized Code to “No”; otherwise the package will contain the default “Optimized Score Code”.

A simple illustration using EM can be done in minutes.

Open EM and a new diagram.
Create a data source for the SAMPSIO.hmeq data set with a target of BAD.
Add the data source to the diagram.
To make the final code less cluttered, in the data source, edit the variables and reject the variables DELINQ, DEROG, JOB, LOAN, MORTDUE, NINQ, and REASON.
Connect the data source to an Impute node from the Modify tab.
Connect the Impute node to a Regression node from the Model tab.
In the Regression node, set the Selection Model property to Stepwise.
Now add two Score nodes and connect the Regression node to both Score nodes.
In the Score(2) node, set the Optimized Code property to “No”.

Your diagram should look like this:

Now run both Score nodes.

Once you’ve run the Score nodes, open the Results viewer for the first Score node. You can see that the regression model only uses two variables – IMP_CLAGE and IMP_DEBTINC – so the creation of the other imputed variables is not required for the regression model. The code in the Optimized SAS Code window does not contain the calculations for the imputation of the unused input variables VALUE, CLNO, and YOJ.

Optimized SAS Code =

*------------------------------------------------------------*;

* TOOL: Imputation;

* TYPE: MODIFY;

* NODE: Impt;

*------------------------------------------------------------*;

label IMP_CLAGE = 'Imputed CLAGE';

IMP_CLAGE = CLAGE;

if missing(CLAGE) then IMP_CLAGE = 179.76627519;

label IMP_DEBTINC = 'Imputed DEBTINC';

IMP_DEBTINC = DEBTINC;

if missing(DEBTINC) then IMP_DEBTINC = 33.779915349;

*------------------------------------------------------------*;

The code in the SAS Code window includes the calculations to impute all the input variables.

SAS Code =

*------------------------------------------------------------*;

* TOOL: Imputation;

* TYPE: MODIFY;

* NODE: Impt;

*------------------------------------------------------------*;

*MEAN-MAX-MIN-MEDIAN-MIDRANGE AND ROBUST ESTIMATES;

label IMP_CLAGE = 'Imputed CLAGE';

IMP_CLAGE = CLAGE;

if missing(CLAGE) then IMP_CLAGE = 179.76627519;

label IMP_CLNO = 'Imputed CLNO';

IMP_CLNO = CLNO;

if missing(CLNO) then IMP_CLNO = 21.296096201;

label IMP_DEBTINC = 'Imputed DEBTINC';

IMP_DEBTINC = DEBTINC;

if missing(DEBTINC) then IMP_DEBTINC = 33.779915349;

label IMP_VALUE = 'Imputed VALUE';

IMP_VALUE = VALUE;

if missing(VALUE) then IMP_VALUE = 101776.04874;

label IMP_YOJ = 'Imputed YOJ';

IMP_YOJ = YOJ;

if missing(YOJ) then IMP_YOJ = 8.9222681359;

*------------------------------------------------------------*;

Where is optimized Score code found?

If you now create model packages for both score nodes, you will find that the package for the Score node with Optimized = Yes displays the Optimized code. The other package for Score(2) displays the SAS Code with all of the variables imputed.

If saving these files from the Results viewer is too interactive, you can use the Score Code Export node or copy the files from the project files. If you navigate to your project folder and open the Workspaces folder, you should find the diagram folder EMWSn. There you will find the folders for your Score nodes. They are named Score or ScoreN, where N is a number. In the Score folder you will find the code displayed in the Results viewer among many internally used files. The file displayed by the Score node Results viewer as “Optimized SAS Code” is named OPTIMIZEDCODE.sas. The file displayed as “SAS Code” is named PATHPUBLISHSCORECODE.sas.

The other .sas files are really just bits and pieces used internally by EM and should not be used for scoring. For example, the PATHFLOWSCORECODE.sas is code used only in the flow. It does calculations for things potentially useful by nodes that may follow the Score node but not needed in scoring like the residuals.

The Optimized SAS Code can in some circumstances represent a large increase in efficiency. In other circumstances there may be little difference. But by default you get both; Code optimized for the model and code that reflects all of the calculations specified in training the model. The selection of which fits your scoring needs could depend on whether or not your scoring process requires values computed but not used in the EM model.

"The order in which the operations shall be performed in every particular case is a very interesting and curious question, on which our space does not permit us fully to enter. In almost every computation a great variety of arrangements for the succession of the processes is possible, and various considerations must influence the selection amongst them for the purposes of a Calculating Engine. One essential object is to choose that arrangement which shall tend to reduce to a minimum the time necessary for completing the calculation."

~ Augusta Ada King, Countess of Lovelace (1815–1852), English mathematician and writer often described as the world's first computer programmer, known for her work on Charles Babbage's early mechanical general-purpose computer

Re: Email ID's from a SAS Dataset in Body

ledfoot — Thu, 28 May 2015 13:50:02 GMT

There has got to be a hundred ways to do this but this is the hardest most obscure way I could think of. The code is incomplete and poorly written, but it runs for me...YOYO

Data WORK.recds_processed;

Length ID $ 6;

ID="1";

output;

ID="2";

output;

ID="3";

output;

ID="4";

output;

ID="5";

output;

ID="6";

output;

ID="7";

output;

ID="8";

output;

ID="9";

output;

ID="10";

output;

ID="11";

output;

ID="12";

output;

ID="13";

output;

ID="14";

output;

ID="15";

output;

ID="16";

output;

ID="17";

output;

ID="18";

output;

ID="1911";

output;

ID="2011";

output;

ID="21";

output;

ID="22";

output;

ID="23";

output;

ID="24";

output;

Run;

%let outfile = idList.txt;

%let outdir = D:\All_Users\sasled\temp;

/* get the count */

Data _NULL_;

Set WORK.recds_processed end=lastobs;

length strcnt $ 20;

retain cnt 0;

if not Missing( ID ) then cnt=cnt+1;

if (lastobs) then do;

/* save cnt */

strcnt = CATS(cnt);

call symput('ID_Cnt', strcnt);

end;

Run;

/* write out the ID info */

Data _NULL_;

Set WORK.recds_processed end=lastobs;

%LET linemax= 80;

length idstr $ 25;

length outlinebuf $ &linemax;

retain row1 1;

retain outlen 0;

retain outlinebuf;

retain fid;

if ( row1 = 1 ) then do; /* open output file */

row1=0;

/* set text file filename */

outf = CATS("&outdir",'\',"&outfile");

put 'Output file name = ' outf;

put ;

rc = Filename("otxt",outf);

if rc ne 0 then do;

msg = sysmsg();

put msg;

call symput('SYSCC','8');

goto Term;

end;

fid = fopen("otxt",'o',,'d');

if (fid <= 0) then do;

/* ERROR: Open failed for file, &outdir\&outfile.*/

emsg=sysmsg();

put "ERROR: " emsg;

call symput('SYSCC','8');

goto Term;

end;

/* output count summary */

outlinebuf = CAT('Number of records processed= ',&ID_Cnt,' the IDs are:');

rc=fput(fid,outlinebuf);

rc=fwrite(fid);

outlinebuf =' ';

end;

idstr = CAT("ID = ",ID);

idlen = LENGTHN(idstr);

if (outlen + idlen + 1) >= &linemax then do;

/* output line on length..*/

rc=fput(fid, outlinebuf);

rc=fwrite(fid);

outlen = 0;

outlinebuf =' ';

end;

outlinebuf = CATX(', ',outlinebuf, idstr);

/* get new buf len */

outlen = LENGTHN(outlinebuf);

if (lastobs) then do;

rc=fput(fid, outlinebuf);

rc=fwrite(fid);

put " Closing output file.....";

rc = fclose(fid);

end;

Term:;

Run;

Scoring Series Part 2: SAS® Enterprise Miner™ Scoring Output Variables

ledfoot — Tue, 06 Oct 2015 15:15:41 GMT

Have you ever wondered what some of the output variables generated by Enterprise Miner score code represent?

The definition of some of variables generated by Enterprise Miner score code may be obvious while others may be less so. The score code published for a model in EM is often a combination of code generated by procedures and by the nodes. The following is a summary of the variables potentially created and a brief description to provide some orientation.

From Procedures

Many of the names of the computed variables (outputs, residuals, etc.,) are created by concatenating a prefix with the name of the corresponding target variable or decision variable. The table below lists most of the possible prefixes for variables calculated in EM procedure score code. If you got really wild with the procedures you might be able to generate some of the more esoteric variables.

List of many of the possible prefixes used for variable names in the EM procedures’ OUT= data sets:

Prefix	Label *	Description
AOV16_	AOV16:VNM	Interval variables binned into 16 equally-spaced groups
BL_	Best Loss:VNM	Best possible loss of any of the decisions
BP_	Best Profit:VNM	Best possible profit of any of the decisions
CL_	Computed Loss:VNM	Loss computed from the target value
CP_	Computed Profit:VNM	Profit computed from the target value
D_	Decision:VNM	Decision chosen by the model
EL_	Expected Loss:VNM	Expected loss of the decision chosen by the model
EP_	Expected Profit:VNM	Expected profit of the decision chosen by the model
E_	Error Function:VNM	Error function
F_	From:VNM	Normalized target value of the category that the case comes from
GRP_	Grouped:VNM	Based on variable characteristics
G_	Grouped:VNM	based on the relationship to the target
IC_	Investment Cost:VNM	Investment cost
IM_	Imputed:VNM	Variable with any missing values replaced
I_	Into:VNM	Normalized category that the case is classified into
M_	Missing:VNM	Missingness indicator dummy variable
P_	Predicted: VNM	Outputs (i.e. predicted values and posterior probabilities)
Q_	Unadjusted P:VNM	Old posteriors, prior to adjustment for priors
RAT_	Anscombe Res.: VNM	Studentized Anscombe residuals
RA_	Anscombe Residual: VNM	Anscombe residuals
RAS_	Anscombe Res.: VNM	Standardized Anscombe residuals
RD_	Residual: VNM	Deviance residuals
RDS_	Dev. Res.: VNM	Standardized deviance residuals
RDT_	Dev. Res.:VNM	Studentized deviance residuals
ROI_	Return on Investment:	Return on investment
RPT_	Pearson Residual: VNM	Studentized Pearson residuals
RP_	Pearson Residual:VNM	Pearson residuals
RPS_	Pearson Res.:VNM	Standardized Pearson residuals
RS_	Residual: VNM	Standardized residuals
RT_	Residual: VNM	Studentized residuals
R_	Residual: VNM	Plain residuals: target minus output
S_	Standard:VNM	Standardized variable
T_	Transform:VNM	Transformed variable
U_	Unnormalized Into:VNM	Un-normalized category that the case is classified into
V_	Validated:VNM	Same as P_ only based on validation data. Tree only.
WOE_	Weight of Evidence:VNM	Relative risk of an attribute or group level

* For non-categorical targets, the "VNM" above indicates the name of the target variable. For categorical targets, "VNM" represents the name of the target variable followed by an equal sign and the un-normalized category value.

The generated score code almost always computes the P_* variable(s), and for a categorical target, the I_* and U_* variable(s). But some modeling engines may allow other ways of fitting categorical targets. For example, Regression (proc DMREG) fits an ordinal target by linear least squares using the index of the category as the actual target value, and hence does not produce posterior probabilities.

Only the decision tree outputs a V_ output variable, which is similar to a corresponding P_ output variable except it is computed using validation data instead of training data.

One of the more ubiquitous variables is the global variable _WARN_. It is used to indicate problems that may occur computing predicted values or making decisions. The _WARN_ variable has 4 columns and each can be set to a specific code.

Column	Code	Description
1	M	Missing input
2	U	Unrecognized input category
3	P	Invalid posterior probability
4	C	Missing cost variable

By default, the EM score code contains no reference to the target variable. Only in the flow score code is the RESIDUAL option specified. So only the EM flow score code can calculate values that depend on the target variable. If the RESIDUAL option is specified in the CODE statement of the modeling procedure, the code should compute the R_* variable(s), and for a categorical target, the F_* variable(s). Other kinds of residuals may be computed if that is feasible, for example CL_*, CP_*, BL_*, BP_*, or ROI_*. Plain residuals are not multiplied by error weights or by frequencies. Plain residuals will always be the actual target value minus the predicted value.

Only if decision processing is specified will variables with prefixes like D_ EL_ or EP_* be calculated. The formula for D_targetname varies with the data mining model. There are too many formulas to list here and they should be identifiable in the score code.

From Nodes

In some cases it is desirable to have an output variable have the same name regardless of the target name. The EM Score node by default provides variables with fixed names for a variety of output variables.

Fixed Output Name	Label	Description
EM_PREDICTION	Prediction for vnm	The prediction variable for an interval target.
EM_PROBABILITY	Probability of Classification	Posterior probability associated with the predicted classification. That is, it corresponds the maximum of the posterior probabilities, max(P1, P2, ..., Pk).
EM_EVENTPROBABILITY	Probability for level n of vnm	Posterior probability associated with target event.
EM_DECISION	Recommended Decision for vnm	Maps to D_targetname variables.
EM_PROFIT	Expected Profit for vnm	Expected profit predicted for a target variable set from EP_targetname
EM_LOSS	Expect Loss for vnm	Expected loss predicted for a target variable set from EL_targetname
EM_CLASSIFICATION	Prediction for vnm	I_variable, the prediction variable for a class target.
EM_SEGMENT	Node or Segment Variable	Segment identifier derived from Decision Tree Leaf, Cluster number, or SOM cell ID

If there is a situation where a different value in a fixed name is required you can use the Rules Builder node to assign just about anything to the EM_Outcome variable.

The Cutoff node will provide the calculated decision point as EM_CUTOFF.

In TwoStage models you also get a variable prefixed with EV_ and labeled “Expected Value:vnm” where vnm = interval target name. It is derived from the predicted value and any specified bias.

Neural Network models can also produce variables named H<number> for the value of the hidden units.

The Link Analysis node provides the item-cluster detection information as the _segment_ variable.

Decision Tree Score code creates variables named _NODE_ and _LEAF_. They are labeled 'Node' and 'Leaf'. They identify each final node or leaf by both leaf number and node number.

The Variable Clustering node can replace a large set of variables with a smaller set of cluster components with little loss of information. The cluster components are named Clusn where "n" is a number.

Some of the Modify nodes in EM can produce output variables with identifying prefixes on the variable name.

Prefix	Node	Description
IMP_	Impute	Original variable’s value or if missing an imputed value
GRP_	Interactive Binning	Group number based on the original variable's value
REP_	Replacement	Replacement values for the variable’s class and interval levels

The Principal Component node by default produces variables named PC_n where "n" is a number but the "PC" can be change in the node properties. Its value is an uncorrelated linear combination of the original input variables.

The Transform Variables node's formula builder defaults to a name like TRANS_n where "n" is a number. However that name can be modified. It does generate names for several pre-defined transformations available from the Train properties, Variables dialog.

The new variables created are named with the selected variable's name and a prefix to identify the specific transformation.

Prefix	Method	Description
BIN_	Bucket	the bin based on the difference between the maximum and minimum values
CNTR_	Centering	the grand mean centered value
EXP_	Exponential	the exponential logarithm of the variable
INV_	Inverse	the inverse of the variable
LOG_	Log	the natural log of the original variable
LG10_	Log 10	the base 10 logarithm of the original variable
OPT_	Optimal Binning	Binned in order to maximize the relationship to the target.
PWR_	Optimal Max. Equalize	Best power transformation to equalize target level spread.
PCTL_	Quantile	values grouped so groups have same frequency in each group
RANGE_	Range Standardization	scaled value of the variable
SQR_	Square	square of the variable
SQRT_	Square Root	square root of the variable.
STD_	Standardize	Produced by subtracting the mean and dividing by the standard deviation.
TI_	Dummy Indicator	creates dummy variable for categorical variables from highest class value to lowest class value

If you are into unsupervised learning you have probably already experimented with the SOM/Kohonen node.

Variable	Label	Description
SOM_Segment	SOM Segment ID	integer identifying the cluster
SOM_ID	SOM ID	contains the row and column in the SOM
Distance	Distance	the distance from each case to the cluster seed
SOM_DIMENSION1	SOM Dimension1	identifies rows or columns in the SOM
SOM_DIMENSION2	SOM Dimension2	identifies rows or columns in the SOM

Incremental Response node provides several variables that could be used to optimize customer targeting.

Variable	Description
EM_P_CONTROL_RESPONSE	Predicted response probability from the control group
EM_P_CONTROL_NONRESPONSE	1- EM_P_CONTROL_RESPONSE
EM_P_ADJ_INCREMENT_RESPONSE	Adjusted to be positive incremental predicted response rate
EM_P_ADJ_INCREMENT_NONRESPONSE	1 - EM_P_ADJ_INCREMENT_RESPONSE
EM_P_ABS_INCREMENT_RESPONSE	Absolute value of the incremental predicted response rate (available when an outcome model used)
EM_P_ABS_INCREMENT_NONRESPONSE	1 - EM_P_ABS_INCREMENT_RESPONSE
EM_P_TREATMENT_RESPONSE	Predicted response probability from the treatment group
EM_P_TREATMENT_NONRESPONSE	1 - EM_P_TREATMENT_RESPONSE
EM_P_INCREMENT_RESPONSE	EM_P_TREATMENT_RESPONSE - EM_P_CONTROL_RESPONSE
EM_P_INCREMENT_NONRESPONSE	EM_P_TREATMENT_NONRESPONSE - EM_P_CONTROL_NONRESPONSE
EM_P_CONTROL_OUTCOME	Predicted value of the outcome variable from the control group
EM_P_TREATMENT_OUTCOME	Predicted value of the outcome variable from the treatment group
EM_P_INCREMENT_OUTCOME	EM_P_TREATMENT_OUTCOME - EM_P_CONTROL_OUTCOME
EM_REV_TREATMENT	Estimated revenue for the treatment group EM_P_TREATMENT_RESPONSE * EM_P_TREATMENT_OUTCOME – Cost or if Constant Revenue is set EM_P_CONTROL_RESPONSE * Revenue_Per_Response – Cost
EM_REV_CONTROL	Estimated revenue for the control group EM_P_CONTROL_RESPONSE * EM_P_CONTROL_OUTCOME
EM_REV_INCREMENT	Estimated incremental revenue EM_REV_TREATMENT - EM_REV_CONTROL

If you can think of other variables generated by EM score code or a better description, please add them to the comments. I left one out just to prove to someone that nobody ever used it.

"Risk comes from not knowing what you're doing."

~ Warren Edward Buffett (born August 30, 1930) an American business magnate, investor, and philanthropist

Scoring Series Part 1

ledfoot — Tue, 06 Oct 2015 17:14:28 GMT

How do you explain scoring to your boss?

What is model scoring vs training?

What is scoring code?

Why is it generated?

What is the goal of analytical modeling?

The answers may depend upon your point of view.

Building a good predictive model is a complex, multi-stage process and is often referred to by the statisticians in our midst as training or fitting a model.

Training a model can involve a variety of different algorithms or ways of examining the data in order to discover implicit relationships. Developing analytical models requires selecting the most appropriate algorithm or combination of algorithms for a given situation. In this, expert knowledge and experience can be critical in developing the very best possible model.

This can lead some statisticians to be quite passionate, even poetic about the elegance of one algorithm compared to another. Such issues can indeed be important. But the person who signs the check for salaries, software and systems doesn’t really care about that. He cares about results.

In the process of analytical modeling, those results come from applying what was learned in training to new data to either classify the new data or provide estimates based on the new data. Another way to think of it is the new data is tested with or compared to the information saved in training and score from that “test” provides the results or scores, hence the euphemism, scoring.

If your situation allows access to the new data that needs to be scored on the same machine where the model was trained it often takes little more than an option or the addition of new node to apply the model. If as is often the case the new data is on a different “production” system that may not have access to all the tools available on the training system you need a different way to apply your model.

There was a time when after all the work of training the model you had to then use the training results often little more that a list of numbers to hand code a program that could run standalone on the “production” system. Of course, after you coded the scoring program you had to test it, and fix it, and test it, etc. With SAS® Enterprise Miner™ all of that is done for you and it produces a fully tested program designed by some of the foremost experts in the field to apply the model or score new data.

The score code generated by Enterprise Miner gives you and easy dependable and accurate way to deploy complex and powerful analytical models on any of your or your customers’ SAS systems regardless of whether or not they have access to Enterprise Miner.

Look for more discussions of scoring in the future. For example, have you ever wondered what some of the output variables generated by Enterprise Miner score code represent?

"Prediction is very difficult, especially if it's about the future."

~ Niels Henrik David Bohr (1885-1962) Danish physicist and Nobel laureate

Tip: How to Migrate Enterprise Miner Projects to a New Operating System

ledfoot — Tue, 06 Oct 2015 17:46:47 GMT

Migrating project files from Enterprise Miner isn’t as simple as you may think—in this case, “copy” just won’t cut it. Fortunately, it is possible to move or migrate all the files and folders in an Enterprise Miner project to different operating system and recreate the project on the new system. You’ll find that this process can be especially helpful when changes to the system architecture take place. However, take note: this article is only concerned with the physical files contained in the project not the project metadata. Metadata migration is usually a SAS system install option.

Before we begin, know that an Enterprise Miner project is composed of many different files and folders and many of which are not portable to a different operating system. This means that the process of moving or migrating an Enterprise project is not as simple as a copy. Luckily, to help improve this rather convoluted and fault-prone process, you can use a SAS Macro utility called the EM_MigrateProject Macro.

Details:

The EM_MigrateProject macro prepares SAS Enterprise Miner project directories and files for movement between operating environments. This includes moving SAS Enterprise Miner projects to or from 32-bit and 64-bit systems, and to or from different Enterprise Miner supported operating systems. Even locale and encoding are converted in this process. The EM_MigrateProject macro also restores prepared projects to a usable form after the directories and files are moved to a new system.

Requirements:

Supported versions include Enterprise Miner 6.1 M2 or later; this includes Enterprise Miner releases based on SAS 9.2 M3 and SAS 9.3 or later.

Limitations:

A project contains both the node results and the node properties that describe how each nodes’ results were achieved. Fortunately, the project migration process preserves both. This means if you examine the Results for each node in a migrated project, they will be exactly the same as the old system. However—as a word of warning—if you run a flow in a migrated project diagram, the results will be replaced with the results from the new system.

Also be aware that each release of Enterprise Miner provides new features and improvements. This means that even if you follow exactly the instructions or properties in a migrated project, you should expect that the training models on the new release will produce models that are significantly different from those produced on the old system.

If you want the new system to create models that are as close as possible to the models from the old system, you will most likely need to adjust the properties for some or all of the nodes in a flow. Usually, this effort is very close to that for creating a new project.

For more information and the macro see SAS Downloads: Enterprise Miner Migrate Project