Imputation modifies the distribution of the input variables. It is something we don't like, but sometimes wee need to live with it. But modifying it in 2 ways (one way for training another way for validation) is even worse I think. As you see, this is not a formal answer, and I admit, I never experimented with the two approaches. If your final goal is to create a predictive model (?), which imputation technique will you use when you do prediction for the unknown cases?  The one derived from the training or from validation dataset? By multiple imputation you mean what PROC MI does? To mix multiple imputation and the use of some validation technique might be difficult. Please someone correct me: - MI is typically used when we have a rather small dataset (with missing values) and we have a (theoretical) model that we want to estimate. - Validation dataset is usually used, when we don't know exactly the model, so we will try a series of models, and use validation dataset to select the best. Typically we have more observations (and also columns) in this case. As a first try I would simply concatenate training and validation datasets and run it. (But not using the target variable.) But you willl need to decide on how to impute using multiple imputation when you predict one unknown case. If you use random forests (or a tree), do you need imputation at all? ... View more

What exactly is in the log? Error? Warning? Nothing? Instead of simply: %put &snapshot_id.; Use this code: %put *****************************; %put &snapshot_id.; %put *****************************; Please also change this line: *%LET snapshot_id=20131231; to: /*%LET snapshot_id=20131231;*/ (this is the right way using comments inside a macro) This way it will be easier to identify the log segment  that you need. Did you know, that your Conditional Start also includes a Pre-Code section? With this code: %let snapshot_id = 20161231; Message was edited by: Gergely Bathó ... View more

Your outer job is called A_testing. Your inner jog is called DIS-JOB-BGIS-108-001. You create a macro variable in the outer job's pre-code. (%let snapshot_id=20171231) What kind of code is in your inner job? Is there for example something like this: %put &snapshot_id.; -and you get a warning about macro reference not found? Are you also using DI parameters in some of the jobs or transformations? A DI parameter is translated to a macro variable. As @Hai.kuo suggested this can hide and outer macro variable definition. When you use Conditional Start/End the code inside the inner job is put inside a macro. If there is a definition for the same macro variable (snapshot_id) this can hide the outer variable. To make the macro variable global use this code: %global snapshot_id; -this code should go in your inner job pre-code section. But I'm not sure it will help, I would need more info about your job. If it is possible, attach  the code that is generated by the outer job. Thanks ... View more

Your code looks fine. 1. Your file (where the macro is saved) is called parcrech.sas (lowcase!)? 2. The file should contain only the macro definition. 3. Make sure the macro has 3 parameters (as in the call of the macro). 4. When you make a failed attempt to auto-load the macro, restart your SAS session. Otherwise it will not try to find it again. Or use: options mrecall; ... View more

I would strive to use the same preprocessing steps on training and validation (and test and all observations to be scored). So doing a separate mean calculation on training and validation is... risky. But to calculate the overall mean using training and validation and then using it in imputation is acceptable. ... View more

If you have a target variable (your final goal is creating a predictive model), then you can use the validation dataset for all your calculations, except the target variable in the validation dataset. This includes calculating means, medians or even a decision tree that predicts an input variable (a model for imputation) or you can use training and validation together to do your favorite dimension reduction (PCA, VarClus, SVD, etc.). The ultimate goal of having a validation dataset is to protection against over-fitting the target. So it can be used to discover structure in the input variables. Do you know of any techniques or best practices, that use validation data to protect against detecting a too complex input space? (For example using validation to determine the optimal number of PCA components.) I'm also interested what others think about this topic. ... View more

Yes. Check your metadata connection settings. proc options group=meta;run; ... View more

Your code should work in EG too. What warning or error messages do you get? Make sure, your if condition is satisfied. Put a  putlog 'CONDITION IS TRUE';  statement above call symput. Do you see this 'CONDITION IS TRUE' message in the log? ... View more

Hi, I made a small test: I used a User Written transformation in the Super Job with this code: %let testMac=XXXXXXXXX; I used a User Written transformation in the Sub Job with this code: %put &testMac.; I have put Sub Job into Super Job surrounded with Conditional Start  and Conditional End. It is working. What is your setup? What version of DI Studio are you using? ... View more

proc options option=sasautos;run; libname sasautos list; But you can extend this search path with: libname mylib 'C:\temp'; options sasautos=(mylib, sasautos); Now you can save a macro called %myfirstmacro with a file name: C:\temp\myfirstmacro.sas ... View more

The join will produce duplicates, if date intervals in the second dataset overlap. On the other hand, if you decide not to include detailed information from this second table, you can aggregate the result of join: proc sql; create table test as select a.id , a.transaction_date , a.group , max(b.group is not missing) as flag from transaction_table a left outer join period_table b on a.transaction_date between b.period_start and b.period_end and a.group = b.group group by 1,2,3 quit; ... View more