DATA Step, Macro, Functions and more

Parse text data into individual words

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 8
Accepted Solution

Parse text data into individual words

Hi SAS community pros!

I am trying to create a data step and/or macro that will parse a large amount of text into individual words for further analysis.

I would like to give SAS a data set with text strings in one variable, and have it return one new variable with all the words from the input variable as separate observations and removing common (or specified list of) delimiters.

I have been experimenting with the SCAN function, but have not had training in how to execute loops or macro array variables.

For example, please see the below excel screenshot for a simplistic example.  Would like to start with SAS data set column A only, and return SAS data set columns A and B

Thank you for your assistance!

parse.png


Accepted Solutions
Solution
‎11-14-2014 01:40 PM
Regular Contributor
Posts: 180

Re: Parse text data into individual words

Based on Reeza solution, how about:

data have;

a="Wolf, meeting with a Lamb astray from the fold,"; n+1; output;

a="resolved not to lay violent hands on him"; n+1; output;

a="but to find some plea"; n+1; output;

run;

data temp(keep=b j);

    set have;

    i=1;

    do while (scan(a, i) ne "");

        b=scan(a,i);

       j+1;

        output;

        i+1;

    end;

run;

proc sql noprint;

  create table want (drop=j) as

  select a,b,j

  from have right join temp

  on n=j

  order by j;

quit;

CTorres

View solution in original post


All Replies
Super User
Posts: 10,490

Re: Parse text data into individual words

This may get you started. This will duplicate the string. COUNTW and SCAN have a large number of possible combinations of delimiters.

data want;

   string = 'For example, please see the below excel screenshot for a simplistic example.';

   words = countw(string,'sp');

   do i=1 to words;

      word = scan(string,i,'sp');

      output;

   end;

   drop i words;

run;

Super User
Posts: 17,801

Re: Parse text data into individual words

Not quite what you asked for, but I'm not sure having the A column as you requested would be useful either.

data have;

a="Wolf, meeting with a Lamb astray from the fold,";output;

a="resolved not to lay violent hands on him"; output;

a="but to find some plea"; output;

run;

data want;

    set have;

    i=1;

    do while (scan(a, i) ne "");

        b=scan(a,i);

        output;

        i+1;

    end;

run;

Solution
‎11-14-2014 01:40 PM
Regular Contributor
Posts: 180

Re: Parse text data into individual words

Based on Reeza solution, how about:

data have;

a="Wolf, meeting with a Lamb astray from the fold,"; n+1; output;

a="resolved not to lay violent hands on him"; n+1; output;

a="but to find some plea"; n+1; output;

run;

data temp(keep=b j);

    set have;

    i=1;

    do while (scan(a, i) ne "");

        b=scan(a,i);

       j+1;

        output;

        i+1;

    end;

run;

proc sql noprint;

  create table want (drop=j) as

  select a,b,j

  from have right join temp

  on n=j

  order by j;

quit;

CTorres

Occasional Contributor
Posts: 8

Re: Parse text data into individual words

Thank you all for your input!  Apologies for the delayed response as I have been sick and travelling.

I am going to use all of your suggestions as a working starting template and improve upon it.  Thanks again!

Frequent Contributor
Posts: 90

Re: Parse text data into individual words

[ Edited ]

Great question, I needed this exact answer today!

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 5 replies
  • 1542 views
  • 12 likes
  • 5 in conversation