BookmarkSubscribeRSS Feed
nikhil_khanolkar
Calcite | Level 5

Hi,

I have a data in SAS which gets updated at the regular interval (Every Hour). I want to update records from the SAS server in the datasets in HDFS and LASR server.

Any help would be appreciated.

Thanks,

Nikhil

5 REPLIES 5
gergely_batho
SAS Employee

Hi,

There are many ways to accomplish this.

By "dataset in HDFS" you mean your data is in SASHDAT format?

What do you mean by "updated"? Do you have new rows, or old rows are changing? Or the whole dataset is completely new?

One example to add (append) observations to a SASHDAT  in-memory file:

data HDATLib  LASRLib.hdatFile(append=yes);

     set sourceLib.sourceFile;

     where recordStatus='NEW';/*put a where statement here that identifies new observations. For example:  &lastLoadTime.  <=  timeStamp*/

run;

After that you can reload the table that is in the LASR memory from the SASHDAT table.

After that you can synchronize the SASHDAT table.


But you could update the rows directly in the LASR memory table (using PROC IMSTAT), then synchronize the associated SASHDAT table.


Message was edited by: Gergely Bathó I was mixing up the table options for SASHDAT and SASIOLA.  Currently  there is NO append=yes table option for the SASHDATengine.

nikhil_khanolkar
Calcite | Level 5

Hi Gergely,

Thanks for Responding.

A) Yes the dataset is in SASHDAT format

B) By update I mean "Old rows are changing"

e.g. Account is the Key variable. We want update the change in Other columns using Key variable Join between SAS and HDFS tables.

       Like Proc SQL update.

Nikhil

gergely_batho
SAS Employee

Do you have an associated in-memory table?

First I would update the in-memory table, then synchronize the SASHDAT table (if needed).

The first task can be done with the PROC IMSTAT UPDATE statement:

SAS(R) LASR(TM) Analytic Server 2.2: Reference Guide

The second task with the PROC IMSTAT SAVE stement:

SAS(R) LASR(TM) Analytic Server 2.2: Reference Guide

gabras
Pyrite | Level 9

Hi, 

i have the same problem but i don't understand how can i solve it.

 

I have table1 which is in memory and table2 which is the table to use for the update. 

table2 can have rows that exist in table1 (update) or new rows (append).

 

How can i use proc imstat update statement in this case?

 

Thank you!!!

gabras
Pyrite | Level 9

To be more precisely:


data table1;
input ID $ 1-2 data $ 3-10;
ID_2=ID+1;
datalines;
1 sysdate
2 sysdate
3 sysdate
4 sysdate
5 sysdate
6 sysdate
7 sysdate
8 sysdate
9 sysdate
10 sysdate
;
run;

 


data table2;
input ID $ 1-2 data $ 3-10;
ID_2=ID+20;
datalines;
1 sysdate1
2 sysdate2
3 sysdate3
;
run;

 

After loading tables in memory (we are in a distributed env) i used this program:


proc imstat data=valibr.table1;
UPDATE data=valibr.table2;
run;
fetch / format ;
quit;

 

Unfortunately it doesen't work. The result is:

Rows from the table
HPS.TABLE1
IDdataID_2
3sysdate323
3sysdate323
3sysdate323
3sysdate323
3sysdate323
3sysdate323
3sysdate323
3sysdate323
3sysdate323
3sysdate323

 

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

Tips for filtering data sources in SAS Visual Analytics

See how to use one filter for multiple data sources by mapping your data from SAS’ Alexandria McCall.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 5 replies
  • 1501 views
  • 0 likes
  • 3 in conversation