SAS Data Integration Studio, DataFlux Data Management Studio, SAS/ACCESS, SAS Data Loader for Hadoop and others

Reading CSV file dynamically into SAS DI

Reply
Occasional Contributor
Posts: 19

Reading CSV file dynamically into SAS DI

Hi,

As a part of project,I got the data in CSV file at a certain path.Now,I want to read this file dynamically into SAS DI Studio to process further.How should I design a job which runs periodically reading this file from the respective location and registers the file in SAS DI with full data initially and then incremental in later runs.

I have an idea of External file Wizard,ODBC connectivity and User written code for complex external files.But,what I want here to completely automate the process of reading file from FTP path into SAS DI by scheduling a job.

Please help.

Thanks,

Maddy

Trusted Advisor
Posts: 3,215

Re: Reading CSV file dynamically into SAS DI

https://communities.sas.com/message/213083#213083 being misplaced? it is the same

---->-- ja karman --<-----
Occasional Contributor
Posts: 19

Re: Reading CSV file dynamically into SAS DI

Jaap,

        Thought this is more relevant to data management and posted here.It is the same.I agree that the physical path may not be hardcoded but I wanted an approach on how to automate the process of reading the external csv file.

When I searched for it,I could only get the process through external file wizard which has to be done manually as in below links.

http://support.sas.com/documentation/cdl/en/etlug/62233/HTML/default/viewer.htm#p18he2u3dngiixn1c6go...

https://support.sas.com/documentation/cdl/en/etlug/62233/HTML/default/viewer.htm#p1h7xnr4n6dmown11ce...

May be if I'm successful in bringing the file from the location into DI through automated process,I can use the below process using file reader to get my job done to populate the data into target table.

http://support.sas.com/documentation/cdl/en/etlug/62233/HTML/default/viewer.htm#p1234xgaebca36n1t9r9...

Please let me know if my requirement is not clear and in case of additional information.

Thanks,

Maddy

Trusted Advisor
Posts: 3,215

Re: Reading CSV file dynamically into SAS DI

You did not touch this one: SAS(R) Data Integration Studio 4.21: User's Guide (Accessing an External File with an FTP Server or an HTTP Server)
SAS is having FTP/SFTP support using their filename. It is limited by not supporting the most (+3 yeras old) options with identification/authentication/encryption. When you need those and needing them by an external program is requiring xcmd access.  The filename and physical name could be placed in the autoexec_usermods of the related app-server. Having the definitions defined it is scheduling as usual.

When you need a trigger for processing the file. Think about segregation of the file-transfer (being send to you) and the processing.
Most schedulers can trigger a job on existence of a file on the local system. Seeing something happening on a system you have no access is not possible.    

---->-- ja karman --<-----
Ask a Question
Discussion stats
  • 3 replies
  • 1650 views
  • 3 likes
  • 2 in conversation