I've never done this in PROC IML, but I have three comments:
1) SAS/IML uses the MODULEI (return void), MODULEIN (return number), and MODULEIC (return char) subroutines to call DLLs that contain C functions. You can pass in vector and matrices for arguments.If you mark an argument as 'update', then the C routine can change the values of the argument (like passing a pointer). I think this is what you want to do.
2) Although I have never done (1), I have called C DLLs from the SAS/IML Studio application, which has a much simpler system for calling DLLs. See the documentation for "Extending IML"-->"UIsing C" at http://support.sas.com/documentation/onlinedoc/imlstudio/WebHelp132/root.htm
3) I have found that well-written vectorized IML code is almost as fast as C code for many applications. You might want to post what you are trying to accomplish. Someone might have IML code that does it already.