topic How do you setup a LASR table to automatically from Hadoop when the LASR server is restarted? in Administration and Deployment

How do you setup a LASR table to automatically from Hadoop when the LASR server is restarted?

rgreen33 — Mon, 06 Feb 2017 12:39:55 GMT

Currently, I have jobs setup and scheduled that will load my Oracle data into Hadoop and then into LASR. Now, I need to set it up so that my LASR table reload automatically (from Hadoop) when my LASR servers are restarted. What is the best way to accomplish this?

By the way, I am new to SAS, so detailed replies are greatly appreciated. 🙂

Thanks,

Ricky

Re: How do you setup a LASR table to automatically from Hadoop when the LASR server is restarted?

JuanS_OCS — Mon, 06 Feb 2017 12:48:44 GMT

Hello,

I guess your data remains on Hadoop, therefore you could use the same scripts to load the data from HDFS to LASR, a few minutes after the restart, to ensure LASR is up and running. Of course, any intermediate script (to check if LASR is up, etc) could help.

Another possible idea, is to save/unload your data from LASR into Hadoop, before stoping them, if you expect the data could have changed, or there might be new tables/queries.

Did you already read this paper? https://support.sas.com/resources/papers/proceedings14/SAS033-2014.pdf

Re: How do you setup a LASR table to automatically from Hadoop when the LASR server is restarted?

rgreen33 — Mon, 06 Feb 2017 12:56:30 GMT

JuanS_OCS,

Thank you for your reply. So, it sounds as if this all has to be done via code...no way to do it via the GUI. So, based on your suggestion above, how do I know when my LASR server has restarted? Would this be a script that is added to my "start-up" steps, which would manually be kicked off when LASR servers are loaded? Or, is there a way to automatically trap for this even and fire this script off on re-start?

Thanks,

Ricky

Re: How do you setup a LASR table to automatically from Hadoop when the LASR server is restarted?

JuanS_OCS — Mon, 06 Feb 2017 13:04:12 GMT

You are welcome @rgreen33,

let me answeryour questions:

There are GUI options (on the library properties in the SAS Management COnsole) for this for the autoload folders, although I believe none for HDFS. Anyway for what you want to achieve, and to get the latest state of your LASR data, I would script it, yes.

To know when LASR was restarted, you have several ways to know it: your logs, on the VA Administrator view, with the SAS Environment Manager... there are even more creative ways, but the main message is that there are many ways to know, fortunately 🙂

As I guess you want to automate as much as possible, I understand you also automated the start-up of the LASR servers with a script, or you want to do it. Therefore the best place to callt his script is to be triggered just after you check that HDFS and LASR are up.