Hi,
I just want to import an URL with SAS that has more den 32767 characters in one line. Does anyone has an idea how I can solve this problem? ...maybe to store the rest in new SAS variables or in new lines?!
Here ist the "normal" code, that truncates after 32767 characters...
%let url2Crawl = http://stores.ebay.de/Fraulein-Frohlich/_i.html?rt=nc%nrstr(%nrstr(&_sid))=1162449139%nrstr(%nrstr(&_trksid))=p4634.c0.m14.l1513%nrstr(%nrstr(&_pgn))=1;
filename crawler3 url "&url2Crawl";
data url(compress=char);
infile crawler3 length=len lrecl=100000 encoding='utf-8';
input lines $varying32767. len;
run;
Log:
NOTE: 153 records were read from the infile CRAWLER3.
The minimum record length was 0.
The maximum record length was 79057.
NOTE: The data set WORK.URL has 153 observations and 1 variables.
NOTE: Compressing data set WORK.URL decreased size by 90.91 percent.
Compressed is 2 pages; un-compressed would require 22 pages.
NOTE: DATA statement used (Total process time):
real time 2.07 seconds
cpu time 0.03 seconds
My problem was solved here:
Thanks.
You will find this post quite interesting.
Simply put you will need to be a bit more creative to scrape website data.
My problem was solved here:
Thanks.
Treat it like a STREAM FILE.
%let url2Crawl = http://stores.ebay.de/Fraulein-Frohlich/_i.html?rt=nc%nrstr(%nrstr(&_sid))=1162449139%nrstr(%nrstr(&_trksid))=p4634.c0.m14.l1513%nrstr(%nrstr(&_pgn))=1;
filename crawler3 url "&url2Crawl";
data url(compress=char);
infile crawler3 recfm=n dlm='<>' dsd encoding='utf-8';
input lines : $2000. @@;
run;
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.