topic Re: Datastep to hadoop - where are my column labels? in SAS Data Management

Datastep to hadoop - where are my column labels?

Filipvdr — Wed, 10 Oct 2018 14:58:17 GMT

Hello, we have written a transformation which will load data to hadoop from sqlserver library.

The problem is that we are losing the labels of my columns. Any idea?

data &hdfsLibref..&outtable (/*debug optie: logupdate=yes */blocksize=64M label="&tableDescr.");
set &_input;
run;

Re: Datastep to hadoop - where are my column labels?

LinusH — Thu, 11 Oct 2018 11:30:54 GMT

The concept of label is local to SAS. Some databases have similar concepts, but they are not supported by SAS/ACCESS engines unfortunately.

One way of enforcing labels is to store them in SAS metadata, and use the META libname engine.

Re: Datastep to hadoop - where are my column labels?

Filipvdr — Thu, 11 Oct 2018 12:54:00 GMT

So you mean we have to use a meta libname statement?

Re: Datastep to hadoop - where are my column labels?

LinusH — Thu, 11 Oct 2018 14:13:49 GMT

"Have to" is a strong statement, but yes (during query time, not necessarily during update).

Re: Datastep to hadoop - where are my column labels?

Filipvdr — Fri, 12 Oct 2018 14:14:20 GMT

thanks, we are trying to load data from Enterprise Guide to Hadoop and from Hadoop to LASR memory. The Eguide and Hadoop/LASR are on different metadata environments.

When I do a copy of the dataset in enterprise guide to my work library with the meta libname statement it copies the labels. Thanks!

But when we are running our code we will connect to the VA environment where we execute some parts of the code. In the VA environment we can run the full libname statement:
LIBNAME MW1DM_05 SQLSVR DATAsrc=bisql2_o SCHEMA=MW1DM_05 AUTHDOMAIN="auth_bisql2_o";

But not this one:
libname MW1DM_05 meta library="MW MV90 DM";

So do we have any other option then add this library to the metadata of our VA environment?

Re: Datastep to hadoop - where are my column labels?

LinusH — Fri, 12 Oct 2018 14:53:31 GMT

So just to be clear, do you have separate metadata server for your EG workspace and VA?
Also, why are you concerned about labels on the external data? As long your corresponding metadata in VA contains labels, that's what the user will see, or am I missing something here?

Re: Datastep to hadoop - where are my column labels?

Filipvdr — Fri, 12 Oct 2018 14:57:23 GMT

yes we have different metadataservers

we load from Eguide to hadoop to LASR. Ofcourse, it's not important what's in hadoop but we also have no labels in the LASR tables.