topic Overflow very long variable from xml file in SAS Programming

Overflow very long variable from xml file

rudfaden — Fri, 27 Nov 2020 13:40:08 GMT

I have an xml file with a variable that runs over 100.000 bytes. I would like it to overflow into several variables. Any ideas on how to do that?

Re: Overflow very long variable from xml file

rhaley1821 — Sun, 29 Nov 2020 20:19:08 GMT

If you are able to import the variable into sas, I would suggest using the SCAN function to break it up!

Here is a similar post with some great examples:

https://stackoverflow.com/questions/47693983/spliting-a-variable-into-multiple-in-sas

Re: Overflow very long variable from xml file

rudfaden — Wed, 02 Dec 2020 09:54:46 GMT

The problem is how to import such a long variable

Re: Overflow very long variable from xml file

Kurt_Bremser — Wed, 02 Dec 2020 10:08:33 GMT

You will have to dissect that data on your own. How you can do that will depend on the layout of the file; if that XML variable is contained on its own single line, in a separate block, or somehow contained within an unbroken stream of data.

Re: Overflow very long variable from xml file

rudfaden — Wed, 02 Dec 2020 10:31:09 GMT

Hi Kurt.

Thanks for your comment. I have given an example on the data structure here. It is this part

that I wan't to import. The text is a resume in plain text.

<CandidateList xmlns="http://schemas.hr-manager.net/restful/2.0/" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<DocumentList>
<Document>
<Id>13289823</Id>
<Name>CV name</Name>
<Extension>pdf</Extension>
<ByteCount>186787</ByteCount>
<DownloadUrl>
https://cdn-recruiter.hr-manager.net/Export/Attachments/ViewDocument.aspx?cid=342&q=KQAAWqKhQYsY6i8aT%2byG16dhiBatjCa7wqDpBB%2flEupzmp3BQQ63DZ41fcyCwWN0
</DownloadUrl>
<Bytes i:nil="true"/>
<Type>Cv</Type>
<TextContent>
very loon text...
</TextContent>
</Document>
</DocumentList>
</CandidateList>

Re: Overflow very long variable from xml file

Kurt_Bremser — Wed, 02 Dec 2020 11:14:17 GMT

See a very simplified example for how I would tackle such an issue:

%let len=20;

data test;
infile datalines truncover;
retain id;
length id 8 text $&len.;
input tag $30.;
select (tag);
  when ("<id>") do;
    input id;
    input;
  end;
  when ('<textcontent>') do;
    input text $&len..@;
    do until (text = '</textcontent>');
      output;
      input text $&len..@;
      if lengthn(text) = 0 then do;
        input;
        input text $&len..@;
      end; 
    end;
    input;
  end;
end;
drop tag;
datalines;
<id>
1
</id>
<textcontent>
xxxxxxxxxxxxxxxxxxxxyyyyyyyyyyyyyyyyyyyzzzzzzzzzzzzzz
</textcontent>
;

Re: Overflow very long variable from xml file

rudfaden — Thu, 17 Dec 2020 17:11:05 GMT

Hi Kurt.

Thanks for yor reply. Forgot to add that the data in textContent is multilined.

<id>
1
</id>
<textcontent>CV
Hi my name is John.

I like to work with sas data.
You can call me on my phone 445-334-566

See you.

John
</textcontent>

Re: Overflow very long variable from xml file

Kurt_Bremser — Sat, 19 Dec 2020 12:54:52 GMT

I have updated the code so that it deals correctly with data that follows the tag on the same line.

%let len=20;

data test;
infile datalines truncover;
retain id;
length id 8 text $&len.;
input line $30.;
tag = scan(line,1,">") !! ">";
line = scan(line,2,">");
select (tag);
  when ("<id>") do;
    input id;
    input;
  end;
  when ('<textcontent>') do;
    if lengthn(line) > 0
    then do;
      text = line;
      output;
    end;
    input text $&len..@;
    do until (text = '</textcontent>');
      output;
      input text $&len..@;
      if lengthn(text) = 0 then do;
        input;
        input text $&len..@;
      end; 
    end;
    input;
  end;
end;
drop tag;
datalines;
<id>
1
</id>
<textcontent>CV
Hi my name is John.

I like to work with sas data.
You can call me on my phone 445-334-566

See you.

John
</textcontent>
;