BookmarkSubscribeRSS Feed
jklaverstijn
Rhodochrosite | Level 12

Dear SAS friends,

 

I am challenged with a conversion of a program from an enviroment where XCMD is allowed to one where it is not. The program creates a zip file with data and a MD5 hash of that data for the validation of integrity of that data once it is being transferred.

 

I have considered a few options:

1) Start a fight to get XCMD allowed. May take forever and succes is not guaranteed.

2) Use CALL MODULExx() using the OpenSSL libcrypto.so library

3) Implement the entire algorithm in datastep or DS2.

4) Ask the community hoping someone has done this already.

 

Of course there may be more.

 

I understand an MD5 hash can be calculated by splitting a message (file) into chunks and adding up the checksums of the chunks. Most solutions seem to involve numeric operations on a 128 bits structure (unsigned int) where we only have 64 bits. So using the MD5() datastep function, however tempting, may not be the solution. Unless of course someone has cracked this.

 

I hope someone can steer me in the right direction.

 

Looking forward to your suggestions,

- Jan.

12 REPLIES 12
RW9
Diamond | Level 26 RW9
Diamond | Level 26

What version control system is on the new system?  SVN at least uses and calculates MD5 as part of its process.  If the version control software doesn't do it automatically, then they must have a import mechanism that does create MD5's to ensure that files uploaded to the new system match what was sent, just get it to store that MD5 and problem solved.  If not and you can't call system, then you have a bit of a problem.

jklaverstijn
Rhodochrosite | Level 12

Hi @RW9,

 

Thanks for chiming in. We do use SVN but it's not going to help here due to the way the existing process is designed. And SVN is only for the development of the software and not for keeping reports and othe output files. And no, we cannot call system; that's why I had to turn to you.

 

Regards Jan.

Sven111
Pyrite | Level 9

This is kind of a shot in the dark since I've never personally tried this and I don't know what it's capabilities or limitations are, but could you use PROC GROOVY or PROC LUA?  There are libraries out there for both for calculating hash functions that you could use without having to code it yourself.  One concern would be if they are available or would have access to the filesystem if XCMD is disabled.  I've been wanting to explore the capabilities of both procs for some time, but haven't had a chance to get around to it.

Sven111
Pyrite | Level 9

Also, does everything need to be done from within SAS and what environment are you running from?  When I've dealt with stuff like this in the past I have usually relied on a shell script to call SAS and then do any post-SAS processing back in the calling script.  This also assumes you're able to run SAS batch jobs from command line rather than just EntGuide.

ChrisHemedinger
Community Manager

Maybe this handy paper from Rick Langston?

 

Implementing Hashing Techniques in SAS®

 

http://support.sas.com/resources/papers/proceedings16/SAS2800-2016.pdf

It's time to register for SAS Innovate! Join your SAS user peers in Las Vegas on April 16-19 2024.
jklaverstijn
Rhodochrosite | Level 12

Very interesting @ChrisHemedinger. Thanks for that, I missed this paper. It does help if I choose to go the way of doing the entire MD5 calculation in SAS. And of course that's where Rick Langston comes out to play as well. At least it appears shows how to do it without the need for the long integers.

 

Regards Jan.

jklaverstijn
Rhodochrosite | Level 12

You're very right Chris, we may do this outside of SAS. It shows how SAS focussed we are that this is not a clearcut approach for us. But may in the end prove the silver bullet.

 

Regards Jan.

jklaverstijn
Rhodochrosite | Level 12

Yes @Sven111 this one caught me off guard. I had the code Groovy prepared, pressed F3 only to find out that proc groovy needs xcmd. I assume the same is true for proc lua. So good idea but no cigar.

 

Regards Jan.

andreas_lds
Jade | Level 19

@jklaverstijn wrote:

Yes @Sven111 this one caught me off guard. I had the code Groovy prepared, pressed F3 only to find out that proc groovy needs xcmd. I assume the same is true for proc lua. So good idea but no cigar.

 

Regards Jan.


Good news: proc lua does not require xmcd, but i don't think that you can use any lib with lua in sas.

Sven111
Pyrite | Level 9

If you can't import libraries, could you just find the source code for one of the hash libraries and copy it directly into PROC LUA and use it that way?

jklaverstijn
Rhodochrosite | Level 12
Thanks for the suggestion. I have found some code in the mean time.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 12 replies
  • 2187 views
  • 0 likes
  • 6 in conversation