Must Reload-on-Start backup storage be loaded manually?

[RogerHed]

Reading the Administrator instructions and activating the VA Reload-on-Start we interpreted that SAS automatically sends backup data when loading data into LASR memory. Seems not to be correct. Just to get it right, do we have to load backup data first and then send this to LASR memory?

[alexal]

@RogerHed ,

Just to get it right, do we have to load backup data first and then send this to LASR memory?

That called the Autoload process.

[RogerHed]

No, we don't use the Autoload. We load data directly to VALIBLA. Applying the Reload-on-start did however not replicated the load to the backup path. The question is if this is supposed to be done by the Reload-on-start or manually?

[alexal]

@RogerHed ,

SAS automatically places a data set copy of the source data in the data provider library that is the designated backing store for the target LASR library. You do not need to do anything manually, just enable Reload-On-Start and load the data to LASR.

[RogerHed]

Great. So this means we have something wrong since the data provider library is not updated on load LASR.
Was curious about the authorization since this is not mentioned in the Admin guide instructions. In Metadata we have same settings on VALIBLA as on the data provider. Running on a Linux. The physical authorization group on the folder may not work. We did a test with a user with physical and Metadata rights to load VALIBLA and Backup directory. But this did not create the backup table anyway.
So anything else we need to consider?

[Madelyn_SAS]

How are you loading the tables? The tables that are eligible to participate in reload-on-start are: 

Participating tables from imports of local files.

Participating tables from imports of Google Analytics, Facebook, or Twitter data.

Additional considerations are here: Additional Considerations 

You could also check the log for reload-on-start. It will be in a location similar to this: 

/Applications/SASVisualAnalytics/VisualAnalyticsAdministrator/Monitoring/Logs

[RogerHed]

Let me clarify. Reload do function as expected when starting up. It's the backup replication that reload doesn't do. As Alexal writes "SAS automatically places a data set copy of the source data in the data provider library", this never happens. Nor is this action logged in any log (perhaps due to suppressed log configuration).

Have checked the .../Monitoring/Logs.

[alexal]

@RogerHed ,

this never happens. 

I would like to review a SAS log from the loading attempt.

[RogerHed]

Attached the log from a manual upload made from VA Administrator (following the example given in How Reload-on-Start Works).

Before this we checked the permissions was correct for the user testing: LASR Server=RM, Library=RM, Folder=RM, Table=RM, R, WM, W and Linux RWX on Group. We also tried to register the backup table before upload if that whould be an issue.

The user that started the LASR do have:

1. The identity that starts the server has metadata-layer access to the table, its parent folder, and its parent library.
2. The identity that starts the server has host access to the table (in the associated data provider library).

We are stuck!

[alexal]

@RogerHed ,

Thanks for your response. Here is what is going on. You are trying to load the data from the HPS library:

proc lasr port=10011
data=HPS.Email_recipients ( label='Inläst torsdagen den 16:e maj 2019 kl. 14:10:48 GMT+0200 från "control.Control.Email_recipients" av "XXX"')
signer="http://XXX:7980/SASLASRAuthorization"
add noclass;
performance
host="XXX"
;
run;

Which was defined as a Base SAS library, located on the server:

NOTE: Libref HPS was successfully assigned as follows:
Engine: V9
Physical Name: /sas/xxx/data/control

As @Madelyn_SAS  said before, The tables that are eligible to participate in reload-on-start are:

• Participating tables from imports of local files.
• Participating tables from imports of Google Analytics, Facebook, or Twitter data.

Here is a piece of SAS code from a loading attempt of the local file. You will see there will be another library called DPPUBLIC. A local file will be imported to that library first (if Reload-On-Start was enabled):

2019-05-16T08:56:39,816 INFO [00000012] :sasdemo - 17 Options VALIDVARNAME=ANY VALIDMEMNAME=EXTEND NO$SYNTAXCHECK;
2019-05-16T08:56:39,817 INFO [00000012] :sasdemo - 18 LIBNAME DPPUBLIC BASE
2019-05-16T08:56:39,817 INFO [00000012] :sasdemo - 18 ! "/<SASConfig/LevX/AppData/SASVisualAnalytics/VisualAnalyticsAdministrator/PublicDataProvider";
2019-05-16T08:56:39,817 INFO [00000012] :sasdemo - NOTE: Libref DPPUBLIC was successfully assigned as follows:
2019-05-16T08:56:39,817 INFO [00000012] :sasdemo - Engine: BASE
2019-05-16T08:56:39,817 INFO [00000012] :sasdemo - Physical Name: /<SASConfig/LevX/AppData/SASVisualAnalytics/VisualAnalyticsAdministrator/PublicDataProvider
2019-05-16T08:56:39,817 INFO [00000012] :sasdemo - 19
2019-05-16T08:56:39,819 INFO [00000012] :sasdemo - 4 The SAS System Thursday, May 16, 2019 08:56:00 AM
2019-05-16T08:56:39,819 INFO [00000012] :sasdemo -
2019-05-16T08:56:39,819 INFO [00000012] :sasdemo - 20 Options VALIDVARNAME=ANY VALIDMEMNAME=EXTEND NO$SYNTAXCHECK;
2019-05-16T08:56:39,819 INFO [00000012] :sasdemo - 21 %let EFI_QUOTED_NUMERICS = yes;
2019-05-16T08:56:39,819 INFO [00000012] :sasdemo - 22 %let EFI_MISSING_NUMERICS = yes;
2019-05-16T08:56:39,819 INFO [00000012] :sasdemo - 23 FILENAME extfile
2019-05-16T08:56:39,819 INFO [00000012] :sasdemo - 23 ! "/tmp/SAS_work0AC6000077DE_lasr.sas.com/SAS_workB32E000077DE_lasr.sas.com/50E1BA44D3DAD124ED98870B9A6E21D
2019-05-16T08:56:39,819 INFO [00000012] :sasdemo - 23 ! D.82ba30d1d01eebde7fb388d12bb52ece_SASServer12_10.csv" ENCODING="UTF-8";
2019-05-16T08:56:39,832 INFO [00000015] :sasdemo - 24 PROC IMPORT DATAFILE=extfile OUT=DPPUBLIC.dp_test
2019-05-16T08:56:39,833 INFO [00000015] :sasdemo - 25 DBMS=CSV REPLACE;
2019-05-16T08:56:39,834 INFO [00000015] :sasdemo - 25 ! DELIMITER=','; DATAROW=2; GUESSINGROWS=502; RUN;

Then from DPPUBLIC to the LASR server:

2019-05-16T08:56:39,965 INFO [00000012] :sasdemo - 78 Options VALIDVARNAME=ANY VALIDMEMNAME=EXTEND NO$SYNTAXCHECK;
2019-05-16T08:56:39,967 INFO [00000020] :sasdemo - 79 DATA LASRLIB.dp_test(LABEL='Imported on Thursday, May 16, 2019 08:56:27 AM GMT-0400 from "dp_test.csv" by "sasdemo"');
2019-05-16T08:56:39,967 INFO [00000020] :sasdemo - 79 ! SET DPPUBLIC.dp_test; RUN;

I hope that makes sense. Let me know if you have any questions.

[RogerHed]

Thank you Alexal!
We are uploading from Library reference CONTROL that is a BASE library. It seems this is reassigned by the code to "libname HPS 'control' loglib=yes;". I know the HPS was tagged in the data provider library. Will check if this is correctly done.

[RogerHed]

Since upload from VA Administrator seems to use a strange method assigning HPS to the SAS BASE library, we created a simple test job in DI Studio. This is our normal way to load data into LASR (VALIBLA) and see Reload-on-Start doesn't create or update the Data Provider (backup).

Providing our settings and the test run from DI Studio Job. As I see it it's very clear data comes from a local SAS BASE Table and not from Metadata My Folder location. The user that started the LASR do have metadata-layer access to the table. The user that starts the server has host access to the table in the associated data provider library (it's new and don't exist in the data provider location yet, both user starting LASR and user running job has write access to local folder).

Do anyone get anything out of this?

[alexal]

@RogerHed ,

My feedback to that will be the same - you are using the wrong loading mechanism. Please open Data Explorer and try to load local data:

[RogerHed]

OK, we tested this approach and it works when true Local Table. We also tested same Importer but with Server Table and it imports but don't create the backup table. Conclusion is that Reload is a general function but the creation of the Reload-data (Data provider copy) is strictly limited to user uploads.

Will put this reflection to our SAS Support and ask reason for not use the Data Provider on other direct loads (as from Batch). Since we use the direct-load-approach in DI Studio instead of Autoload, we need the Batch code to replicate to the Data provider in order to reload correct data version on server or service restarts.

[alexal]

@RogerHed ,

. Conclusion is that Reload is a general function but the creation of the Reload-data (Data provider copy) is strictly limited to user uploads.

Yes, that is correct. Like we mentioned a few times before, not all tables can participate in reload-on-start. Please read the following page: Reload Methods.