file not found

[p12937]

I have a sas job that reads a csv and writes out to another csv. Then a second sas job reads this new csv. We have these jobs scheduled to run every 3 hours in windows task scheduler (first one at 10 past the hour and the second one at 15 past). Sometimes the second job does not find the new file even though I can see it in the folder. It seems like windows is not refreshed or something like that. Has anyone experienced this? Thanks.

[PaigeMiller]

Let's start with basic debugging:

Have you confirmed that the first job has finished writing the CSV before the second job executes? How do you know?

Have you confirmed that the file name is created properly to be read by the second job?

If that isn't it, show us the exact error in the SAS log of the second job.

Can you change the jobs from separated jobs, to one job where the two SAS programs are appended to each other, so the second part can't possibly start running before the first part end. Or better yet, if you can combine the programs, don't write to CSV, save the results of job 1 in a SAS data set and then use that in job 2.

[p12937]

Confirmation of completion: dos window closes and sas log shows that job is complete.

File name is correct. I run the second job again and it finds and processes the file.

There is no error message, the second job executes a windows dir command and looks for that file and 0 records are returned. When I run the job again it finds the file.

[p12937]

we are creating a second file because the original file has carriage returns and line feed that need to be removed.

[PaigeMiller]

---

@p12937 wrote:

Confirmation of completion: dos window closes and sas log shows that job is complete.

File name is correct. I run the second job again and it finds and processes the file.

There is no error message, the second job executes a windows dir command and looks for that file and 0 records are returned. When I run the job again it finds the file.

---

Is there some reason why this has to be two jobs? Could these two programs be run sequentially in a single job? If so, could you use a SAS data set instead of writing a CSV?

[p12937]

we are creating a second file because the original file has carriage returns and line feeds that need to be removed.

[PaigeMiller]

---

@p12937 wrote:

we are creating a second file because the original file has carriage returns and line feeds that need to be removed.

---

I don't see this as a justification for separate programs that launch separately.

[Tom]

---

@p12937 wrote:

Confirmation of completion: dos window closes and sas log shows that job is complete.

File name is correct. I run the second job again and it finds and processes the file.

There is no error message, the second job executes a windows dir command and looks for that file and 0 records are returned. When I run the job again it finds the file.

---

Sounds like disk timing issue.  The second job is starting before the operating system has finished really writing the file and updating the directory entries.  Try just adding a delay between the two steps. If you can edit the SAS code you could either put a sleep() function call at the end of the first one or the start of the second one.

[PaigeMiller]

There is no error message, the second job executes a windows dir command and looks for that file and 0 records are returned.

Have you checked to see that the first program is actually writing the correct contents to the csv file? If this isn't happening correctly, then the timing isn't the problem. Incorrect contents in the csv file could result in 0 records being read, and no error message might appear.

[Kurt_Bremser]

What do you mean by "I can see it in the folder"? How do you verify that the file is there? Do you work with shared resources on a network?

Anyway, inspect the timestamp of the file; you might find it was created after the second job started. If you use a scheduler, use conditions to prevent a follow-up job from starting too early.

[p12937]

maybe I should try a windows community blog.

[p12937 · Accepted solution]

the second job runs 5 minutes after jobs. Is there maybe a windows command that could refresh the directory/cache or something? we are going to try running the windows dir command multiple times

[Kurt_Bremser]

Have you looked at the timestamps of the files, as suggested? Also look at the timestamps of the log files, as they tell you exactly when the SAS process closed the file.

I have experienced examples of SAS processes doing literally nothing for minutes because of system congestion.

[p12937]

yes, I have checked file timestamps, log timestamps and log messages. I really don't think this is a SAS issue, but since I am running SAS programs I thought someone might have had a similar issue with files not being available right away on a windows server or sfa.

[PaigeMiller]

Did you check to see if the csv file that is created has the proper contents? If it is not created with the proper contents, this could cause the second program to read 0 records.