Setting DI job options at job level instead of transformation

[LaurieF]

My latest DI transformation takes a JSON file as input, but there are potentially multiple files to read, from which I'm extracting just one dataset. The size of the file doesn't matter, because the JSON engine is very quick at reading even a multi-million observation file.

For ease of management, I'm calling the transformation multiple times in a job, with the output work dataset from the transformation being the JSON entry. It works, and works very well.

I previously had the JSON file as File or directory (input) option in the transformation, but with multiple transformation calls in a single job (I have reasons) I have put that file reference in as a job parameter. And that works fine. What doesn't is that I don't seem to. be able to reference that parameter from the job - I have to go into the parameter and change it instead of having it as a prompt that I can easily edit.

Interestingly when I set up the parameter, I can use Test Prompts… in the same way as I would a transformation option. I just don't seem to be able to reference it any other way. Is there something obvious I have missed?

[Patrick]

I don't fully understand what your problem with the parameter is but... have you considered using a loop transformation for calling your "json read" transformation parameterized so you need to define the transformation only once? A loop transformation would also allow you to execute the reads in parallel.

[LaurieF]

A loop wouldn't be appropriate - each file has different metadata which the transformation acts upon. Remember the issue is the name of the json file, not the entries within it.

When I open and run the job from within DI, I want to select the JSON file, as I would if I were running it from just a transformation. 

From a transformation I open it, click on Code options and select the file I want. From the job, I have to open the parameter, edit it and change the default to the file I want. There's no Code options at job level.

[Patrick]

DI Studio is a development tool for ETL jobs. Running jobs directly out of DIS is for unit testing but your end-goal must be to deploy the jobs and run them in batch.

Whatever you develop via DIS must be fit for batch processing where there is no user interaction.

If you want something with user interaction more on an adhoc bases then use EG/SAS Studio or eventually implement a Stored Process.

[LaurieF]

I've been SASsplained! What an ignorant and patronising reply. And wrong. I've been using SAS for 45 years, and DI for twenty. I know  what it can do. I know it can be used, at least as well as anybody else.

Now if you aren't going to explain to me how I can use a facility built into a SAS/DI job, which doesn't appear to me to work, I suggest you delete what you just wrote and go and SASsplain to someone else.

[Patrick]

It's sometimes really hard to tell on which expertise level someone is just based on a few sentences in a forum. Sorry if I got yours wrong. 

May-be I misunderstand but wouldn't that just be a one-time config? So why is this an issue?