SAS Data Integration Studio, DataFlux Data Management Studio, SAS/ACCESS, SAS Data Loader for Hadoop and others

How to pass macro parameter to File reader transformation to read latest csv file?

Reply
Contributor SYN
Contributor
Posts: 39

How to pass macro parameter to File reader transformation to read latest csv file?


I have one situation,

I have 3 different .CSV files and using file reader tranformation I have build the job for all 3 csv files.

I want to pass latest files every time using my user written code.

Can you please help me how to pass macro parameters.

Is there any simple way please let me know.

Thanks in advance.

Contributor SYN
Contributor
Posts: 39

Re: How to pass macro parameter to File reader transformation to read latest csv file?

How to pass macro parameter to File reader transformation to read latest csv file?

Please I have this question in SAS DI 3.4.

Thank you.

Super Contributor
Posts: 408

Re: How to pass macro parameter to File reader transformation to read latest csv file?

Hi,

DI 3.4 may not support my approach; I haven't checked. What I do in 4.x, in a nutshell, is to

1) define the CSV in the metadata with a macro variable reference as the filename  (like &csvfile). make certain the generated code wraps this in double quotes or you're stuffed. This may actually be the breaking point for 3.4.

2) define and load a table that holds files present in your input location and keeps track of their processed status, eg through a timestamp.

3) Then in a job process that table in the first step to find out which file must be processed. Stick that value in a macro var. As the next step process the external file you defined in the metadata. Give that step a parameter to transfer the result from step 1 into the macro variable that matches the filename of the CSV file from 1). As a last step (or a subsequent job) mark the file as processed in the table as defined in 2).

Good luck, Jan.

Super Contributor
Posts: 349

Re: How to pass macro parameter to File reader transformation to read latest csv file?

Hi SYN,

Check this link..it might be helpfull

https://communities.sas.com/message/129750#129750

This will help you in making a parameter file and you can resolve that in you DI,One more thing you need to do is to get the latest file.

Thanks,

Shiva

Ask a Question
Discussion stats
  • 3 replies
  • 469 views
  • 0 likes
  • 3 in conversation