File Size Limitation in XML Mapper

[Babloo]

When I read a 55K records from XML file via SAS XML Mapper to create a .map file, it prompts an error message "The file size exceeds the recommended maximum. ......".

Can Mapper read only small XML files < 5K or less?

[BrunoMueller]

Have a look at this Usage note http://support.sas.com/kb/39/804.html

It explains various ways how to handle this situation.

I usually recommend to make the XML file smaller and build the map file.

[Babloo]

Could you please help me understand how can I achieve the following?

1. Cut the XML file where the elements repeat and map this file, which is a subset of the larger XML file.
2. If you have the schema, create an XMLMap file from the schema rather than the XML file.

[BrunoMueller]

for 1

Assume the following XML

<?xml version="1.0" encoding="iso-8859-1" ?>
<NHL>
  <CONFERENCE> Eastern 
    <DIVISION> Southeast  
      <TEAM name="Thrashers"  abbrev="ATL" />  
      <TEAM name="Hurricanes" abbrev="CAR" />  
      <TEAM name="Panthers"   abbrev="FLA" />  
      <TEAM name="Lightning"  abbrev="TB" />  
      <TEAM name="Capitals"   abbrev="WSH" />  
   </DIVISION>
 </CONFERENCE> 

 <CONFERENCE> Western
   <DIVISION> Pacific  
     <TEAM name="Stars"   abbrev="DAL" />  
     <TEAM name="Kings"   abbrev="LA" />  
     <TEAM name="Ducks"   abbrev="ANA" />  
     <TEAM name="Coyotes" abbrev="PHX" />  
     <TEAM name="Sharks"  abbrev="SJ" />  
   </DIVISION>
  </CONFERENCE> 
</NHL>

<TEAM .../> is the repeating element that will make up one observation

we only need one <TEAM ... /> entry to build the XML map file, the others can be deleted.

Also <CONFERENCE> and <DIVISION> appear more than once, but we only need them once

For 2

If you have a XSD you can build the map file from this

File -> Open XML Schema

Tools -> Automap using XSD

the Map file created might not be what you want

[Babloo]

I'm trying to understand the first part of your response. In the example below, although <TEAM .../> is the repeating element, each of the element seem to have different record and I'm not certain how can I make it to one observation. Appreciate if you could guide me here.

[Babloo]

<?xml version="1.0" encoding="iso-8859-1" ?>
<NHL>
  <CONFERENCE> Eastern 
    <DIVISION> Southeast  
      <TEAM name="Thrashers"  abbrev="ATL" />  
      <TEAM name="Hurricanes" abbrev="CAR" />  
      <TEAM name="Panthers"   abbrev="FLA" />  
      <TEAM name="Lightning"  abbrev="TB" />  
      <TEAM name="Capitals"   abbrev="WSH" />  
   </DIVISION>
 </CONFERENCE> 

 <CONFERENCE> Western
   <DIVISION> Pacific  
     <TEAM name="Stars"   abbrev="DAL" />  
     <TEAM name="Kings"   abbrev="LA" />  
     <TEAM name="Ducks"   abbrev="ANA" />  
     <TEAM name="Coyotes" abbrev="PHX" />  
     <TEAM name="Sharks"  abbrev="SJ" />  
   </DIVISION>
  </CONFERENCE> 
</NHL>

In the above example, could you please help me understand how will you cut the repeating elements?

[BrunoMueller]

Like so

<?xml version="1.0" encoding="iso-8859-1" ?>
<NHL>
  <CONFERENCE> Eastern 
    <DIVISION> Southeast  
      <TEAM name="Thrashers"  abbrev="ATL" />  
   </DIVISION>
 </CONFERENCE> 
</NHL>

[BrunoMueller]

Have a go at creating the map file automatically, see the doc for more information

Importing an XML Document Using the AUTOMAP= Option to Generate an XMLMap

[Babloo]

Per the document ,the following SAS statements import the XML document Nhl.xml. In my case, I do not have .map file to get it done. Could you please guide me how to proceed now?

filename nhl 'C:\Example\Nhl.xml'; /*1*/
filename map 'C:\Output\NhlGenerate.map'; /*2*/

libname nhl xmlv2 automap=replace xmlmap=map; /*3*/

proc print data=nhl.team; /*4*/
run;

[BrunoMueller]

there is no way around reading the documentation!

check the description for /*3*/

[Babloo]

Whether the 'team' refers to a SAS dataset below?

proc print data=nhl.team; /*4*/
run;

[Babloo]

I tried the following code, but I'm not finding the .map file in /wrk1. There is no error or warning in the log file. Appreciate if you could guide me here to generate .map file.

LIBNAME test "/shared/sas" ;
filename testing "/wrk1/dynamic_new.xml"; /*1*/
filename map "/wrk1/dynamic_new.map"; /*2*/

libname testing xml xmlmap=map; /*3*/

data dynamic; 
set test.DYNAMIC(obs=1000); 
run; 

[BrunoMueller]

Which SAS version are you using?

Run this code and provide the log

%put NOTE: &=sysscpl;
%put NOTE: &=sysvlong;

Your example code differs quite a bit from the example in the doc:

for instance your are missing the automap=replace

you are also using the XML versus the XMLV2 engine

Any reasons for the changes?

Run Proc CONTENTS on the testing library to see which tables are available, then you can use a DATA Step to read the actual data.

[Babloo]

Here you go.

23         GOPTIONS ACCESSIBLE;
24         %put NOTE: &=sysscpl;
NOTE: &=sysscpl
25         %put NOTE: &=sysvlong;
NOTE: &=sysvlong

I'm using SAS 9.2 and if I use automap=replace and XMLV2 I used to receive an error.

33         proc contents data=testing._all_ nods;
34         run;

ERROR: Physical file does not exist, /wrk1/dynamic_new.map.

[BrunoMueller]

So I see, the AUTOMAP= option is available beginning with SAS® 9.3 TS1M2, so it is not available to you.

You have to go to the approach of reducing the size of your XML file as explained earlier and then build the map from it.

If you know what you want to read from the XML files you can build the map file yourself with an text editor.

Some editors, like Notepad++, have support to actually look at the structure, you may also use your browser to look at the content of the xml file to figure out the structure.

On linux system you might have the following command 

xmllint –format file.xml

available to make an XML easier to read