topic Re: Error with DQ functions on sas code in SAS Data Management

Error with DQ functions on sas code

Kal_ETL — Wed, 23 Oct 2019 18:33:06 GMT

Hello,

I am migrating a couple of jobs and codes from sas 9.3 to sas 9.4. One of the codes is using data quality functions like this:

%dqload(dqlocale=(ESMEX), dqsetuploc="/<qkb path>/dqsetup.txt");

filename names '/<customized qkb path>/scheme/ESMEX_namess.sch.bfd';

After that there is a data step where functions like this are called:

name_1=dqstandardize(upcase(nombre),'ESMEX_CHAR_NAMES');

name_2=dqschemeapply(upcase(nombre_1),'nompal','BFD');

Name_STD1=dqschemeapply(upcase(nombre_2),'esacen','BFD');

Also there are calls to:

- dqparse

- dqmatch

- dqidentify

- dqparsetokenget.

All codes and jobs works fine in sas 9.3, now I am starting to test in 9.4 I am unable to start with this first code. I must say it is running from command line in a Unix environment and all the services are running (started). The errors in the log file are:

%dqload(dqlocale=(ESMEX), dqsetuploc="/<qkb path>/dqsetup.txt");

NOTE: DQLOAD macro beginning.

WARNING: No QKB metadata could be found.

NOTE: DQLOAD macro ending.

And linew below, on the same log file:

INFO: Character variables have defaulted to a length of 200 at the places given by: (Line):(Column). Truncation can result.
      150:1    name_1
      151:1    name_2
ERROR: Internal error detected in function DQSTANDARDIZE.  The DATA step is terminating during the EXECUTION phase.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: SAS set option OBS=0 and will continue to check statements. This might cause NOTE: No observations in data set.
NOTE: There were 1 observations read from the data set LIBRARY.DATASETNAME

What I did was to copy all the folder for the qkb from sas 9.3 and put on a path in sas 9.4, and also add the path to the app.cfg for this Data Management Server installation like this:

qkb/path = /opt/sas/qkb/ci/26/<subfolders for customized qkb>

In dqsetup also did some modifications to the original file:

DISK <dataquality>\<subfolders for customized qkb>\chopinfo;
DISK <dataquality>\<subfolders for customized qkb>\grammar;
DISK <dataquality>\<subfolders for customized qkb>\locale;
DISK <dataquality>\<subfolders for customized qkb>\phonetx;
DISK <dataquality>\<subfolders for customized qkb>\regexlib;
DISK <dataquality>\<subfolders for customized qkb>\scheme;
DISK <dataquality>\<subfolders for customized qkb>\vocab;

Am I missing something? Please if anyone have experience on this, help can be awesome. Do i need to register something on metadata?

Thank you for your help.

Re: Error with DQ functions on sas code

SASKiwi — Wed, 23 Oct 2019 19:26:46 GMT

I suspect QKB's are not the same between SAS versions and simply copying from one version to another won't work. Are these QKB customizations you want to import into 9.4? If so I'm sure there would be a process for migrating these although I don't have time right now to research this.

Re: Error with DQ functions on sas code

ShayneGrant — Wed, 23 Oct 2019 19:52:26 GMT

Older QKBs should still work in newer versions of SAS. I don't see any obvious error nor do I see and error message which helps understand what the issue is. You could try not using dqsetup.txt as that is an older way to specify a QKB, you just specify the path to the root of the QKB. So for example:

%dqload(dqlocale=(ESMEX), dqsetuploc="<qkb path>");

If this doesn't work, then I recommended contacting Technical Support

Re: Error with DQ functions on sas code

Kal_ETL — Tue, 29 Oct 2019 19:45:56 GMT

Thank you @ShayneGrant and @SASKiwi for your help, both were a great help. At the end:

- I remved the txt file you mentioned.

- In sas.cfg file, on the opt installation:

- Add qkb path (For some reason if I don't add, it works but I think it is a good practice to do it).

- Modify qkb/compatver

- Change to yes qkb/allow_incompat

- The most important change was to add rwxr-xr-x to all the imported QKB, since for some reasson sub folders and files in it doesn't inherit changes to up level folders.

Now I have another question but I will create a new post for it.

Grettings