Hi everyone,
I am trying to extract data from my own LinkedIn page, using SAS 9.4.
I've seen the various posts on this community and internet, for instance
https://blogs.sas.com/content/sasdummy/2017/12/04/scrape-web-page-data/
I've used the PWENCODE procedure to encode my password into a txt file. I get it back in the macro variable &PASS.
So far, I've written the code :
filename recupFIC "C:\PASS to output file\test.xml"; proc http method="GET" url="https://www.linkedin.com/company/MYSITENUMBER/admin/analytics" out=recupFIC WEBAUTHDOMAIN="www.linkedin.com" webusername="my username here" webpassword="&PASS." ; run;
When I run this code, a window opens and I am asked to fill the fields for a metadata server : Server Name, User Id and password. I don't understand where I can find these informations.
When I try to scrap a simple page (no authentification required), it works perfectly.
Do you have any idea?
Thank you very much in advance,
WEBAUTHDOMAIN is for an administered SAS mid-tier, so that's not an option you need.
WEBUSERNAME and WEBPASSWORD is for "Basic Auth" -- but LinkedIn does not use that mechanism. 3rd party applications must use LinkedIn APIs and connect with OAuth2 -- a much more complex negotiation. And I'm not sure that LinkedIn APIs provide the data you want to get. Check their Developer site to see what's possible.
Web scraping is most likely against LinkedIn's data use policy. While you might be just trying to experiment with your own profile, taking it further is probably against their rules. If you just want to "practice" parsing your page, use your web browser to Save As HTML and then use SAS to process that as an INFILE.
Chris
WEBAUTHDOMAIN is for an administered SAS mid-tier, so that's not an option you need.
WEBUSERNAME and WEBPASSWORD is for "Basic Auth" -- but LinkedIn does not use that mechanism. 3rd party applications must use LinkedIn APIs and connect with OAuth2 -- a much more complex negotiation. And I'm not sure that LinkedIn APIs provide the data you want to get. Check their Developer site to see what's possible.
Web scraping is most likely against LinkedIn's data use policy. While you might be just trying to experiment with your own profile, taking it further is probably against their rules. If you just want to "practice" parsing your page, use your web browser to Save As HTML and then use SAS to process that as an INFILE.
Chris
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.