Hi everyone,
I am trying to extract data from my own LinkedIn page, using SAS 9.4.
I've seen the various posts on this community and internet, for instance
https://blogs.sas.com/content/sasdummy/2017/12/04/scrape-web-page-data/
I've used the PWENCODE procedure to encode my password into a txt file. I get it back in the macro variable &PASS.
So far, I've written the code :
filename recupFIC "C:\PASS to output file\test.xml"; proc http method="GET" url="https://www.linkedin.com/company/MYSITENUMBER/admin/analytics" out=recupFIC WEBAUTHDOMAIN="www.linkedin.com" webusername="my username here" webpassword="&PASS." ; run;
When I run this code, a window opens and I am asked to fill the fields for a metadata server : Server Name, User Id and password. I don't understand where I can find these informations.
When I try to scrap a simple page (no authentification required), it works perfectly.
Do you have any idea?
Thank you very much in advance,
WEBAUTHDOMAIN is for an administered SAS mid-tier, so that's not an option you need.
WEBUSERNAME and WEBPASSWORD is for "Basic Auth" -- but LinkedIn does not use that mechanism. 3rd party applications must use LinkedIn APIs and connect with OAuth2 -- a much more complex negotiation. And I'm not sure that LinkedIn APIs provide the data you want to get. Check their Developer site to see what's possible.
Web scraping is most likely against LinkedIn's data use policy. While you might be just trying to experiment with your own profile, taking it further is probably against their rules. If you just want to "practice" parsing your page, use your web browser to Save As HTML and then use SAS to process that as an INFILE.
Chris
WEBAUTHDOMAIN is for an administered SAS mid-tier, so that's not an option you need.
WEBUSERNAME and WEBPASSWORD is for "Basic Auth" -- but LinkedIn does not use that mechanism. 3rd party applications must use LinkedIn APIs and connect with OAuth2 -- a much more complex negotiation. And I'm not sure that LinkedIn APIs provide the data you want to get. Check their Developer site to see what's possible.
Web scraping is most likely against LinkedIn's data use policy. While you might be just trying to experiment with your own profile, taking it further is probably against their rules. If you just want to "practice" parsing your page, use your web browser to Save As HTML and then use SAS to process that as an INFILE.
Chris
Are you ready for the spotlight? We're accepting content ideas for SAS Innovate 2025 to be held May 6-9 in Orlando, FL. The call is open until September 25. Read more here about why you should contribute and what is in it for you!
Learn how use the CAT functions in SAS to join values from multiple variables into a single value.
Find more tutorials on the SAS Users YouTube channel.