Hi,
I am currently working on an automatic data refresh using Windows Task Scheduler. Every night at midnight web scraped data will be uploaded on my computer. I will then run the automated SAS code for data manipulation purposes. However I would only like to run the SAS code on the data that is new. My file size is very large, so I would like to reduce process time by only running the code through the new data rather than the entire data set. I was wondering if there is a code that would only run on new data.
Thank you!
Sure, just add a step to your process so that it extracts the new data only and then runs on that. Exactly how depends a bit more on your data structure. There needs to be a relatively clean way to know what’s new data.
@bklassen wrote:
Hi,
I am currently working on an automatic data refresh using Windows Task Scheduler. Every night at midnight web scraped data will be uploaded on my computer. I will then run the automated SAS code for data manipulation purposes. However I would only like to run the SAS code on the data that is new. My file size is very large, so I would like to reduce process time by only running the code through the new data rather than the entire data set. I was wondering if there is a code that would only run on new data.
Thank you!
is there date and timestamp on the records? you could setup a step or selection statement for processing records using those indicators.
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 16. Read more here about why you should contribute and what is in it for you!
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.
Select SAS Training centers are offering in-person courses. View upcoming courses for: