Downloading a CSV using Proc HTTP

Jacob3 — Fri, 22 Jul 2022 14:03:14 GMT

I am trying to download a CSV file directly from a website. The file is about 1.5 GB (roughly 5.5 million rows).

This works for 1 GB which corresponds to 3,600,716 observations. I then tried just downloading the file from the website directly to my computer. It keeps restarting the download at exactly 1 GB.

Any ideas on how I can pull the full file?

I use the following code:

filename out "c:\temp\ratings.csv";

proc http
url="http://ratingshistory.info/csv/20220601%20Standard%20&%20Poor's%20Ratings%20Services%20US%20Public.csv"
method="get" out=out;
run;

The log is:

ERROR: Request Failed and number of retries has been exceeded.
NOTE: PROCEDURE HTTP used (Total process time):
real time 4:02.51
cpu time 22.67 seconds

NOTE: 200 OK
NOTE: The SAS System stopped processing this step because of errors.

Re: Downloading a CSV using Proc HTTP

ChrisHemedinger — Fri, 22 Jul 2022 14:18:16 GMT

I gave this a quick try and I see the same behavior. I think this might be a problem with the server that hosts these files. Smaller files seem to work fine.

topic Re: Downloading a CSV using Proc HTTP in SAS Programming

Downloading a CSV using Proc HTTP

Re: Downloading a CSV using Proc HTTP