hi,
is there a way to download information froma webpage into sas?
fo example, if I go to a youtube video:
https://www.youtube.com/watch?v=kQ1L39be1e0
is it possible to download say the number of views and the number of likes at any given moment?
thank you!
HTML is text.
Go to the page in question and view the page source. See if you can find the elements in the page source. If you can find it then yes you can, and then you can determine how to parse the file to get the information of interest.
Hi Reeza,
On the web page I right click - inspect on the views count and got that element in html. But when I try to parse it in sas I have trouble. Is there some documentation on how to parse html elements?
Thank you!
Not really, but as I mentioned it's a text file, you process it the same way. You can search using FIND/INDEX functions or PRX functions.
Also, the solution from Tom Kari, in your other question, had samples of how that was occurring.
SAS isn't a good webscraping tool...it can be used, but there are better tools out there...Import.IO is free, either web or desktop based and has a free version. It can be scripted so that you have results run and available on a regular basis.
Hi LinusH,
thanks for the link!
In the meanwhile I just tried to do some code which will extract the number of views of the youtube video Kitten Meets Computer:
data kitten;
length view $32767;
filename the_vid url "https://www.youtube.com/watch?v=kQ1L39be1e0";
infile the_vid lrecl=32767;
input;
view = _infile_;
run;
data kitten2;
set kitten;
IF
(INDEX(view,'watch-view-count') = 0) THEN DELETE;
run;
data kitten3;
set kitten2;
place=index(view,'watch-view-count');
like=substr(view,place+18,5);
drop place;
run;
Just in the kitten3 I should do the part "like=substr(view,place+18,5);" dynamic because once the number of views reaches 10,000 the 5 will become 6 etc.
Also, post a screenshot or sample of the text you're trying to parse.
It's hard to make suggestions otherwise.
This is a really nice application for regular expressions. Try this code instead of your data steps for kitten2 and kitten3:
data kitten2;
retain re;
if _n_ = 1
then re=prxparse('#(<div class="watch-view-count">)([\d,]+)(</div>)#');
set kitten;
if prxmatch(re, view)
then do;
view_count = input(prxposn(re, 2, view), comma20.);
output;
end;
run;
The regular expression (assignment to re) has three parts, each in a set of parentheses:
First is the literal "<div class="watch-view-count">", and third is the literal "</div>"
Second, represented by "[\d,]+", looks for one or more repetitions of a digit or a blank.
PRXPOSN returns the string found as the second part, as a character variable, which is converted to a number by the INPUT function.
Tom
hi,
I actually counted the number of spaces, I know, not the most efficient way...
I tried meanwhile scraping other web pages. I tried to scrap the friends that liked a post that I uploaded on my faceboof, but it didn't work --> sas didn't download any of the html elements associated with the opening sliding window with the people who liked.
I also saw an example from the paper that you suggested that scraps job postings from the Edmonton career website. There the actual scraping is done with Perl, and the parsing with SAS, so I guess that for more complicated wed scraping SAS is not the best program.
I also saw import.io and tried it online and it seemed to be a bit too limited because it chooses itself what to scrap, and sometimes it didn't scrap anything.
Thank you!
@ilikesas wrote:
hi,
I actually counted the number of spaces, I know, not the most efficient way...
I tried meanwhile scraping other web pages. I tried to scrap the friends that liked a post that I uploaded on my faceboof, but it didn't work --> sas didn't download any of the html elements associated with the opening sliding window with the
I also saw import.io and tried it online and it seemed to be a bit too limited because it chooses itself what to scrap, and sometimes it didn't scrap anything.
Thank you!
1. Yes, change that to a scan or find and you'll probably be able to make it dynamic
2. For FB you definitely need to go through the API, same with Twitter. Here's a SAS paper that tried it years ago, and most likely won't work today : http://www.sascommunity.org/wiki/Social_Networking_and_SAS:_Running_PROCs_on_Your_Facebook_Friends
3. Import.Io -> the user can select the fields to parse. In general download app is better than the web one.
Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. Registration is now open through August 30th. Visit the SAS Hackathon homepage.
Register today!Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.
Find more tutorials on the SAS Users YouTube channel.