topic INPUT - How to read a single line which is longer than 32767 characters? in SAS Programming

INPUT - How to read a single line which is longer than 32767 characters?

HendrikE — Mon, 13 Jun 2016 08:19:59 GMT

Hi,

I just want to import an URL with SAS that has more den 32767 characters in one line. Does anyone has an idea how I can solve this problem? ...maybe to store the rest in new SAS variables or in new lines?!

Here ist the "normal" code, that truncates after 32767 characters...

%let url2Crawl = http://stores.ebay.de/Fraulein-Frohlich/_i.html?rt=nc%nrstr(%nrstr(&_sid))=1162449139%nrstr(%nrstr(&_trksid))=p4634.c0.m14.l1513%nrstr(%nrstr(&_pgn))=1;

filename crawler3 url "&url2Crawl";

data url(compress=char);
	infile crawler3 length=len lrecl=100000 encoding='utf-8';
	input lines $varying32767. len;
run;

Log:

NOTE: 153 records were read from the infile CRAWLER3.
      The minimum record length was 0.
      The maximum record length was 79057.
NOTE: The data set WORK.URL has 153 observations and 1 variables.
NOTE: Compressing data set WORK.URL decreased size by 90.91 percent.
      Compressed is 2 pages; un-compressed would require 22 pages.
NOTE: DATA statement used (Total process time):
      real time           2.07 seconds
      cpu time            0.03 seconds

Re: INPUT - How to read a single line which is longer than 32767 characters?

RW9 — Mon, 13 Jun 2016 08:36:13 GMT

You will find this post quite interesting.

https://communities.sas.com/t5/General-SAS-Programming/How-To-Fetch-Data-From-Website-Through-SAS-Code/m-p/274057/highlight/true#M38102

Simply put you will need to be a bit more creative to scrape website data.

Re: INPUT - How to read a single line which is longer than 32767 characters?

HendrikE — Mon, 13 Jun 2016 09:15:43 GMT

My problem was solved here:

https://communities.sas.com/t5/CoDe-SAS-German/Einlesen-von-mehr-als-32767-Zeichen/gpm-p/276862#M2032

Thanks.

Re: INPUT - How to read a single line which is longer than 32767 characters?

Ksharp — Mon, 13 Jun 2016 09:40:51 GMT

Treat it like a STREAM FILE.

%let url2Crawl = http://stores.ebay.de/Fraulein-Frohlich/_i.html?rt=nc%nrstr(%nrstr(&_sid))=1162449139%nrstr(%nrstr(&_trksid))=p4634.c0.m14.l1513%nrstr(%nrstr(&_pgn))=1;

filename crawler3 url "&url2Crawl";

data url(compress=char);
	infile crawler3 recfm=n dlm='<>' dsd encoding='utf-8';
	input lines : $2000. @@;
run;