DATA Step, Macro, Functions and more

INPUT - How to read a single line which is longer than 32767 characters?

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 6
Accepted Solution

INPUT - How to read a single line which is longer than 32767 characters?

Hi,

 

I just want to import an URL with SAS that has more den 32767 characters in one line. Does anyone has an idea how I can solve this problem? ...maybe to store the rest in new SAS variables or in new lines?!

 

Here ist the "normal" code, that truncates after 32767 characters...

 

%let url2Crawl = http://stores.ebay.de/Fraulein-Frohlich/_i.html?rt=nc%nrstr(%nrstr(&_sid))=1162449139%nrstr(%nrstr(&_trksid))=p4634.c0.m14.l1513%nrstr(%nrstr(&_pgn))=1;

filename crawler3 url "&url2Crawl";

data url(compress=char);
	infile crawler3 length=len lrecl=100000 encoding='utf-8';
	input lines $varying32767. len;
run;

Log:

 

NOTE: 153 records were read from the infile CRAWLER3.
      The minimum record length was 0.
      The maximum record length was 79057.
NOTE: The data set WORK.URL has 153 observations and 1 variables.
NOTE: Compressing data set WORK.URL decreased size by 90.91 percent.
      Compressed is 2 pages; un-compressed would require 22 pages.
NOTE: DATA statement used (Total process time):
      real time           2.07 seconds
      cpu time            0.03 seconds

Accepted Solutions
Solution
‎06-13-2016 05:15 AM
Occasional Contributor
Posts: 6

Re: INPUT - How to read a single line which is longer than 32767 characters?


All Replies
Super User
Super User
Posts: 7,392

Re: INPUT - How to read a single line which is longer than 32767 characters?

You will find this post quite interesting.

https://communities.sas.com/t5/General-SAS-Programming/How-To-Fetch-Data-From-Website-Through-SAS-Co...

 

Simply put you will need to be a bit more creative to scrape website data.

Solution
‎06-13-2016 05:15 AM
Occasional Contributor
Posts: 6

Re: INPUT - How to read a single line which is longer than 32767 characters?

Super User
Posts: 9,662

Re: INPUT - How to read a single line which is longer than 32767 characters?

Treat it like a STREAM FILE.

 

 

%let url2Crawl = http://stores.ebay.de/Fraulein-Frohlich/_i.html?rt=nc%nrstr(%nrstr(&_sid))=1162449139%nrstr(%nrstr(&_trksid))=p4634.c0.m14.l1513%nrstr(%nrstr(&_pgn))=1;

filename crawler3 url "&url2Crawl";

data url(compress=char);
	infile crawler3 recfm=n dlm='<>' dsd encoding='utf-8';
	input lines : $2000. @@;
run;
☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 3 replies
  • 277 views
  • 0 likes
  • 3 in conversation