SAS Functions called in FCMP Functions from DS2 Packages Error: "Freeing Memory Not Allocated Pool"

[CurtisMackWSIPP]

Hi,

This is a complicated one, so I really appreciate anybody who gets through it.

I am learning DS2 and FCMP and how they interact.  I have found problems with many Data Step functions (I have tested STRIP TRIM and LENGTH), if used in an FCMP package which is then referenced in a DS2 package.   If a method is defined that uses the function, but it is not called in DS2, the code will run, but then crash with the a dump starting with "ERROR: Freeing Memory Not Allocated Pool [000000000E782200] Value [000000000E391640] tkFnc pool."  If the DS2 method using the function is actually called, that method does not run and I get "ERROR: Invalid instruction" followed by the previously mentioned error.  It will also frequently crash the SAS session.  This happens in traditional SAS and EG, although EG doesn't show the dump.

I have tested using the SAS functions (Trim) directly in the DS2 method and it works fine.  I also still get the error if the FCMP functions don't have parameters and just use a hardcoded string in the function.

I don't know if this is a bug, or if I am completely missing the way these new features work.  I do a lot of .NET programming and am really hoping for an object oriented approach to SAS programming.  I would love to ditch MACRO code as much as possible.

Here is an example piece of code.  My application is much more complicated, but it boils down to this.  This code works as is.  If you uncomment the commented sections in the code you should get the errors.

libname sas_ds2 "C:\Temp\ggg";
libname base '.';

proc fcmp outlib=base.fcmpsubs.sql_tools ;
function without_trim($ myVal) $ 100;
length outvar $ 100;
outvar = myVal;
return(outvar);
endsub;

function with_trim($ myVal) $ 100;
length outvar $ 100;
outvar = trim(myVal);
return(outvar);
endsub;

run;
quit;

proc ds2;
package sql_tools / overwrite=yes language='fcmp' table='base.fcmpsubs';
run;
package sas_ds2.dataset_info/overwrite=yes;
dcl package sql_tools st();
method getUnTrimmed() returns char(100);
declare char(100) tempStr;
tempStr = st.without_trim('String to Trim ');
return tempStr;
end;
* If this is uncommented, you should get the crash dump after the code runs;
/*
method getTrimmed() returns char(100);
declare char(100) tempStr;
tempStr = st.with_trim('String to Trim ');
return tempStr;
end;
*/
run;

data _null_ ;
declare package sas_ds2.dataset_info f;
declare char(100) varlistOut;
method init();
f = _new_ sas_ds2.dataset_info();
varlistOut = f.getUnTrimmed();
put varlistOut;
* If this and the previous block are uncommented, this step
does not run and the you get a crash dump;
/*
varlistOut = f.getTrimmed();
put varlistOut;
*/
end;
enddata;
run;
quit;

[ChrisNZ]

---

crash with the a dump starting with "ERROR: Freeing Memory Not Allocated Pool [000000000E782200] Value [000000000E391640] tkFnc pool." 

---

When you start seeing that sort of message, an email to Tech Support is not far behind....

[CurtisMackWSIPP]

I have submitted this to SAS support.  I will update this post with what I learn from them.

[ballardw]

---

@CurtisMackWSIPP wrote:

Hi,

This is a complicated one, so I really appreciate anybody who gets through it.

I am learning DS2 and FCMP and how they interact.  I have found problems with many Data Step functions (I have tested STRIP TRIM and LENGTH), if used in an FCMP package which is then referenced in a DS2 package.   If a method is defined that uses the function, but it is not called in DS2, the code will run, but then crash with the a dump starting with "ERROR: Freeing Memory Not Allocated Pool [000000000E782200] Value [000000000E391640] tkFnc pool."  If the DS2 method using the function is actually called, that method does not run and I get "ERROR: Invalid instruction" followed by the previously mentioned error.  It will also frequently crash the SAS session.  This happens in traditional SAS and EG, although EG doesn't show the dump.

I have tested using the SAS functions (Trim) directly in the DS2 method and it works fine.  I also still get the error if the FCMP functions don't have parameters and just use a hardcoded string in the function.

I don't know if this is a bug, or if I am completely missing the way these new features work.  I do a lot of .NET programming and am really hoping for an object oriented approach to SAS programming.  I would love to ditch MACRO code as much as possible.

Here is an example piece of code.  My application is much more complicated, but it boils down to this.  This code works as is.  If you uncomment the commented sections in the code you should get the errors.

libname sas_ds2 "C:\Temp\ggg";
libname base '.';

proc fcmp outlib=base.fcmpsubs.sql_tools ;
function without_trim($ myVal) $ 100;
length outvar $ 100;
outvar = myVal;
return(outvar);
endsub;

function with_trim($ myVal) $ 100;
length outvar $ 100;
outvar = trim(myVal);
return(outvar);
endsub;

run;
quit;

proc ds2;
package sql_tools / overwrite=yes language='fcmp' table='base.fcmpsubs';
run;
package sas_ds2.dataset_info/overwrite=yes;
dcl package sql_tools st();
method getUnTrimmed() returns char(100);
declare char(100) tempStr;
tempStr = st.without_trim('String to Trim ');
return tempStr;
end;
* If this is uncommented, you should get the crash dump after the code runs;
/*
method getTrimmed() returns char(100);
declare char(100) tempStr;
tempStr = st.with_trim('String to Trim ');
return tempStr;
end;
*/
run;

data _null_ ;
declare package sas_ds2.dataset_info f;
declare char(100) varlistOut;
method init();
f = _new_ sas_ds2.dataset_info();
varlistOut = f.getUnTrimmed();
put varlistOut;
* If this and the previous block are uncommented, this step
does not run and the you get a crash dump;
/*
varlistOut = f.getTrimmed();
put varlistOut;
*/
end;
enddata;
run;
quit;

---

I would probably start by assigning an actual location to the Library Base. Not saying that will fix things but it may reduce issues with possibly looking in the "wrong" place.

I don't see any Options cmplib statement to tell SAS where compiled functions reside.

[CurtisMackWSIPP]

That was a really good suggestion, but unfortunately, it did not work.

I think this functionality is handled by the DS2 line "package sql_tools / overwrite=yes language='fcmp' table='base.fcmpsubs';"

I am assuming the OPTIONS statement is only needed if calling these functions from open SAS.  Please correct me if I am wrong.

[jimbarbour]

I've not used DS2 except once or twice just to see if it might be a better option than a traditional Data step for a particular project.  My experiments with DS2 have uniformly been disappointing.  A traditional Data step is almost always faster unless you have a process that is highly CPU bound.  If it helps, Troy Martin Hughes wrote a PharmaSUG paper that discusses when DS2 actually runs faster than a traditional Data step.  When I attended WUSS 2018, one of the speakers said that to his knowledge DS2 really hasn't become popular and that not very many people are using it.  My sense is that DS2 has a very narrow range in which it is actually a better option than a traditional Data step or Proc SQL.  Incidentally, parallel processing can be done with traditional Data steps either via an rSubmit or via a SYSTASK.  However, I think you are interested in DS2 because of it's syntax rather than in its performance.

So, I don't have vast experience from which to draw on re DS2, but I can confirm that an error results.  See log below.  You may want to do as @ChrisNZ suggests and open a ticket with SAS.

String to Trim                                                                                      
ERROR: Access violation
ERROR: General error
NOTE: PROC DS2 has set option NOEXEC and will continue to prepare statements.
84         quit;

NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE DS2 used (Total process time):
      real time           0:00:00.28
      cpu time            0:00:00.28
      Timestamp           September 03, 2020 12:52:30

Jim

[CurtisMackWSIPP]

Just an update.  SAS Tech has been working on this.  It looks like they are working on a Hot Fix for M7. I will post the number when I get it.

[CurtisMackWSIPP · Accepted solution]

SAS has created a hot fix I9S001 for this bug.  

You can find it at 

https://tshf.sas.com/techsup/download/hotfix/HF2/I9R.html#I9S001

I installed it using the standard (and inexcusably difficult) hotfix process.  This hotfix is flagged with a "D" in the analysis step which means it needs additional manual steps.  My reading of the docs lead me to the conclusion that this is only important if you are in a "Deployed" installation.  Since I use a standalone installation of SAS, I did no additional steps and it installed.  Note that this hotfix is actually a bundle of two hotfixes I9N001 and I9P001, so you will not see I9S001 listed in some of the output.

I have tested my original program and it now works correctly! 

Thanks SAS support for the relatively fast patch!