Using DataFlux. In standalone mode, meaning no other SAS environment is available.
I need to make sure there are files in a folder (name changes as the export date is appended to the filename)
before the Text File Import node is executed to prevent a "Error reading file" error.
My plan was to put an expression in the Process Flow checking to make sure at least one file exists.
then use If node to branch to the import process for that set of files .
I see in the expression user manual the fileexists function but nothing to allow passing dynamic names to it.
I thought about calling a batch file to return the DIR contents but cant seem to get the external node to work either.
Thank you
Quite tough to provide you with the most consistent approach. However:
Therefore if there is one file in your folder, it is possible to read the folder, get the file name and pass it to the text input
You can also activate the FILENAME option for retrieving an additional column storing the name of the file.
Hi
Are you able to know the file name or is there any green flag stating that the file is present and ready for being process?
Vincent
The problem is the filenames are different each time we process.
I prefer the Text input node doesn't even get executed if there is no file in the folder .
Ok, it still possible to read the content of the folder before starting your process. What would be the rules for identifying the file to read?
Cheers
Each week we get new files from one customer but once a month we get files from another customer. the files are currently in separate folders as I couldn't figure a way to only load cust_a files in TextNode A and cust_b in TextNode B pointing to same folder .. SO thus separate folders. Back to main question. on Week 1,2,3 there would only be files in one folder .. on week 4 we would have both sets of files. I preferred to have both processes started at once in one Process flow so the user only has one process to start.
When started it would
check that the input folder had files, if so branch to the actual import , cleanse and export .
if cust_b folder was empty it would branch to a complete section and wait for the other customer files to finish without erroring and stopping the process. BUT it should stop if there really was an error in one of the steps during load and cleanse.
I Think I might be able to do it with the other solution you provided on another question ( the listener demo )
Quite tough to provide you with the most consistent approach. However:
Therefore if there is one file in your folder, it is possible to read the folder, get the file name and pass it to the text input
You can also activate the FILENAME option for retrieving an additional column storing the name of the file.
Vincent,
Thank you for the details. this sounds like what I have so far. I do use the macro variables. I split all the files into separate folders based on structure and I do use just the path. I also use the FILENAME feature to propagate the name through the process. I think based on your description and the similarities I am probably at the best solution for what I have. Since the files are large the header check process in the beginning IS slow as it has to read the entire file to strip out the header. I have requested python to be added to our environment so I think I will use that at the start of the process (external application node) to validate files exist for processing, verify header / column structure hasn't changed and then let the processes proceed.
Thank you again for taking the time to help and validate I'm on the right track.
SAS Innovate 2025 is scheduled for May 6-9 in Orlando, FL. Sign up to be first to learn about the agenda and registration!
Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.
Find more tutorials on the SAS Users YouTube channel.