DATA Step, Macro, Functions and more

File Size Limitation in XML Mapper

Reply
PROC Star
Posts: 633

File Size Limitation in XML Mapper

When I read a 55K records from XML file via SAS XML Mapper to create a .map file, it prompts an error message "The file size exceeds the recommended maximum. ......".

Can Mapper read only small XML files < 5K or less?

SAS Super FREQ
Posts: 830

Re: File Size Limitation in XML Mapper

Have a look at this Usage note http://support.sas.com/kb/39/804.html

 

It explains various ways how to handle this situation.

 

I usually recommend to make the XML file smaller and build the map file.

PROC Star
Posts: 633

Re: File Size Limitation in XML Mapper

Posted in reply to Bruno_SAS

Could you please help me understand how can I achieve the following?

 

  1. Cut the XML file where the elements repeat and map this file, which is a subset of the larger XML file.

     

  2. If you have the schema, create an XMLMap file from the schema rather than the XML file.
SAS Super FREQ
Posts: 830

Re: File Size Limitation in XML Mapper

for 1

Assume the following XML

<?xml version="1.0" encoding="iso-8859-1" ?>
<NHL>
  <CONFERENCE> Eastern 
    <DIVISION> Southeast  
      <TEAM name="Thrashers"  abbrev="ATL" />  
      <TEAM name="Hurricanes" abbrev="CAR" />  
      <TEAM name="Panthers"   abbrev="FLA" />  
      <TEAM name="Lightning"  abbrev="TB" />  
      <TEAM name="Capitals"   abbrev="WSH" />  
   </DIVISION>
 </CONFERENCE> 

 <CONFERENCE> Western
   <DIVISION> Pacific  
     <TEAM name="Stars"   abbrev="DAL" />  
     <TEAM name="Kings"   abbrev="LA" />  
     <TEAM name="Ducks"   abbrev="ANA" />  
     <TEAM name="Coyotes" abbrev="PHX" />  
     <TEAM name="Sharks"  abbrev="SJ" />  
   </DIVISION>
  </CONFERENCE> 
</NHL>

<TEAM .../> is the repeating element that will make up one observation

we only need one <TEAM ... /> entry to build the XML map file, the others can be deleted.

 

Also <CONFERENCE> and <DIVISION> appear more than once, but we only need them once

 

For 2

If you have a XSD you can build the map file from this

File -> Open XML Schema

Tools -> Automap using XSD

 

the Map file created might not be what you want

PROC Star
Posts: 633

Re: File Size Limitation in XML Mapper

Posted in reply to Bruno_SAS

I'm trying to understand the first part of your response. In the example below, although <TEAM .../> is the repeating element, each of the element seem to have different record and I'm not certain how can I make it to one observation. Appreciate if you could guide me here.

PROC Star
Posts: 633

Re: File Size Limitation in XML Mapper

Posted in reply to Bruno_SAS

 

 

<?xml version="1.0" encoding="iso-8859-1" ?>
<NHL>
  <CONFERENCE> Eastern 
    <DIVISION> Southeast  
      <TEAM name="Thrashers"  abbrev="ATL" />  
      <TEAM name="Hurricanes" abbrev="CAR" />  
      <TEAM name="Panthers"   abbrev="FLA" />  
      <TEAM name="Lightning"  abbrev="TB" />  
      <TEAM name="Capitals"   abbrev="WSH" />  
   </DIVISION>
 </CONFERENCE> 

 <CONFERENCE> Western
   <DIVISION> Pacific  
     <TEAM name="Stars"   abbrev="DAL" />  
     <TEAM name="Kings"   abbrev="LA" />  
     <TEAM name="Ducks"   abbrev="ANA" />  
     <TEAM name="Coyotes" abbrev="PHX" />  
     <TEAM name="Sharks"  abbrev="SJ" />  
   </DIVISION>
  </CONFERENCE> 
</NHL>

In the above example, could you please help me understand how will you cut the repeating elements?

 

SAS Super FREQ
Posts: 830

Re: File Size Limitation in XML Mapper

Like so

 

<?xml version="1.0" encoding="iso-8859-1" ?>
<NHL>
  <CONFERENCE> Eastern 
    <DIVISION> Southeast  
      <TEAM name="Thrashers"  abbrev="ATL" />  
   </DIVISION>
 </CONFERENCE> 
</NHL>
SAS Super FREQ
Posts: 830

Re: File Size Limitation in XML Mapper

Have a go at creating the map file automatically, see the doc for more information

Importing an XML Document Using the AUTOMAP= Option to Generate an XMLMap

PROC Star
Posts: 633

Re: File Size Limitation in XML Mapper

Posted in reply to Bruno_SAS

Per the document ,the following SAS statements import the XML document Nhl.xml. In my case, I do not have .map file to get it done. Could you please guide me how to proceed now?

 

filename nhl 'C:\Example\Nhl.xml'; /*1*/
filename map 'C:\Output\NhlGenerate.map'; /*2*/

libname nhl xmlv2 automap=replace xmlmap=map; /*3*/

proc print data=nhl.team; /*4*/
run;
SAS Super FREQ
Posts: 830

Re: File Size Limitation in XML Mapper

there is no way around reading the documentation!

check the description for /*3*/
PROC Star
Posts: 633

Re: File Size Limitation in XML Mapper

Posted in reply to Bruno_SAS

Whether the 'team' refers to a SAS dataset below?

 

proc print data=nhl.team; /*4*/
run;

 

PROC Star
Posts: 633

Re: File Size Limitation in XML Mapper

Posted in reply to Bruno_SAS

I tried the following code, but I'm not finding the .map file in /wrk1. There is no error or warning in the log file. Appreciate if you could guide me here to generate .map file.

 

LIBNAME test "/shared/sas" ;
filename testing "/wrk1/dynamic_new.xml"; /*1*/
filename map "/wrk1/dynamic_new.map"; /*2*/

libname testing xml xmlmap=map; /*3*/

data dynamic; 
set test.DYNAMIC(obs=1000); 
run; 
SAS Super FREQ
Posts: 830

Re: File Size Limitation in XML Mapper

Which SAS version are you using?

 

Run this code and provide the log

%put NOTE: &=sysscpl;
%put NOTE: &=sysvlong;

Your example code differs quite a bit from the example in the doc:

for instance your are missing the automap=replace

you are also using the XML versus the XMLV2 engine

 

Any reasons for the changes?

 

Run Proc CONTENTS on the testing library to see which tables are available, then you can use a DATA Step to read the actual data.

PROC Star
Posts: 633

Re: File Size Limitation in XML Mapper

Posted in reply to Bruno_SAS

Here you go.

 

23         GOPTIONS ACCESSIBLE;
24         %put NOTE: &=sysscpl;
NOTE: &=sysscpl
25         %put NOTE: &=sysvlong;
NOTE: &=sysvlong

I'm using SAS 9.2 and if I use automap=replace and XMLV2 I used to receive an error.

 

33         proc contents data=testing._all_ nods;
34         run;

ERROR: Physical file does not exist, /wrk1/dynamic_new.map.
SAS Super FREQ
Posts: 830

Re: File Size Limitation in XML Mapper

So I see, the AUTOMAP= option is available beginning with SAS® 9.3 TS1M2, so it is not available to you.

 

 

You have to go to the approach of reducing the size of your XML file as explained earlier and then build the map from it.

 

If you know what you want to read from the XML files you can build the map file yourself with an text editor.

 

Some editors, like Notepad++, have support to actually look at the structure, you may also use your browser to look at the content of the xml file to figure out the structure.

 

On linux system you might have the following command 

xmllint –format file.xml

available to make an XML easier to read

 

 

Ask a Question
Discussion stats
  • 19 replies
  • 281 views
  • 3 likes
  • 3 in conversation