topic Re: XML Engine Performance in SAS Programming

XML Engine Performance

Saranya_Sub — Fri, 15 May 2020 14:18:57 GMT

Hi All,

I have a requirement to convert a ODM xml file to sas dataset. i have used below method to do that, but the execution time is more as it is a 25MB file. Is there any way to do it faster.

filename NHL "/path/ODM.xml";

filename map "/path/odm_map.map";

libname NHL xmlv2 xmlmap=map;

proc copy in=nhl out=work;

run;

ODM.xml is a 25 mb file so when i try to copy those datasets present in NHL to work library, it's taking allmost 10 min, is there any way to reduce the execution time.

Re: XML Engine Performance

ChrisNZ — Sat, 16 May 2020 02:19:06 GMT

Use option fullstimer and show us the log.

Re: XML Engine Performance

Saranya_Sub — Tue, 19 May 2020 07:28:25 GMT

Hi Chris,

Thanks for the response . Please find below details of the fullstimer when i tried to load ~3mb XML File

Total Table created : 54 in which more than 14 tables have ~18000 records and remaining tables have ~1000 records

NOTE: PROCEDURE COPY used (Total process time):
real time 4:03.72
user cpu time 4:00.68
system cpu time 1.17 seconds
memory 32842.37k
OS Memory 52888.00k
Timestamp 19/05/2020 03:14:43 PM
Step Count 4 Switch Count 802
Page Faults 3
Page Reclaims 17172
Page Swaps 0
Voluntary Context Switches 3662
Involuntary Context Switches 27536
Block Input Operations 504
Block Output Operations 56512

SAS Version : 9.4_M2

total memsize is MEMSIZE=3221225472 in our server

Thanks,

Saranya

Re: XML Engine Performance

ChrisNZ — Tue, 19 May 2020 09:38:29 GMT

The process is clearly CPU-bound. I am unsure whether proc copy can be replaced with a more efficient method, but another way would be to either procure a faster CPU, or split the process into pieces and spread it across several CPUs or cores.