topic Re: How to identify whether the value of column has 5 or more consonants without vowel? in SAS Programming

How to identify whether the value of column has 5 or more consonants without vowel?

nikunjgattani — Wed, 11 Apr 2018 14:59:57 GMT

Hi,

I have a string column named "Column_Test" which cotains text up to 200 characters.
I want to identify those rows/ids where "Column_Test" value has 5 or more consonants without vowel.

Could anyone help me on this ?

Thanks
Nikunj

Re: How to identify whether the value of column has 5 or more consonants without vowel?

tomrvincent — Wed, 11 Apr 2018 15:04:17 GMT

Load the column into an array, looping one character at a time. Then, count the members of the array that you consider consonants.

Re: How to identify whether the value of column has 5 or more consonants without vowel?

novinosrin — Wed, 11 Apr 2018 15:10:39 GMT

Hope this helps:


data have;
input Column_Test $50.;
datalines;
ABCEDIFOGUHJKLMNPQRSTVXZ
;

data want;
set have;
want=length((compress(Column_Test,'AEIOU','L')));
if want>5 then put '5 or more consonants without vowel';
run;

Re: How to identify whether the value of column has 5 or more consonants without vowel?

PaigeMiller — Wed, 11 Apr 2018 15:12:50 GMT

Use the COUNTC function to find if there are voewls. Then use length to count the length of the string;

data want;
     y = 'dfktbcpl';
     nvowels=countc(y,'aeiouy');
     length=length(y);
run;

Re: How to identify whether the value of column has 5 or more consonants without vowel?

Astounding — Wed, 11 Apr 2018 15:27:16 GMT

How will you decide whether the letter "Y" is a vowel or not?

Re: How to identify whether the value of column has 5 or more consonants without vowel?

ballardw — Wed, 11 Apr 2018 15:37:47 GMT

Spaces? Punctuation? Special Characters such as @#$%^&*()_-=+.

Is this supposed to be contiguous consonants or to the above not "reset" the count.

Might want to provide some concrete examples and whether they meet the requirement or not.

Re: How to identify whether the value of column has 5 or more consonants without vowel?

nikunjgattani — Wed, 11 Apr 2018 15:56:50 GMT

Objective is to find mistyping

Ex -

looking more for things like strkct instead of strict

Re: How to identify whether the value of column has 5 or more consonants without vowel?

nikunjgattani — Wed, 11 Apr 2018 15:58:39 GMT

Objective is to find mistyping

Ex -

looking more for things like strkct instead of strict

Yes, we have to reset the counter once it got vowel in the string and again start counting.

We have to ignore spaces, punctuation and special character.

Re: How to identify whether the value of column has 5 or more consonants without vowel?

Reeza — Wed, 11 Apr 2018 16:02:33 GMT

There used to be a dictionary or spelling procedure. I don't know if it still exists.

Re: How to identify whether the value of column has 5 or more consonants without vowel?

Astounding — Wed, 11 Apr 2018 16:53:27 GMT

So for the exceptions, like "strengths" you are willing to inspect them to see if they are valid? (You still need to decide whether "Y" qualifies as a vowel. Is "tryst" a valid word or a mis-spelling? How about "encrypts"?)

Re: How to identify whether the value of column has 5 or more consonants without vowel?

Reeza — Wed, 11 Apr 2018 17:16:29 GMT

I think you're better off doing a dictionary search. You can find lists online. Flag anything not a valid word and then build your rules from there.

Re: How to identify whether the value of column has 5 or more consonants without vowel?

nikunjgattani — Wed, 11 Apr 2018 18:08:37 GMT

Hi,

Here we are not looking for a correct word but a mistyped word. Text
contains various places name too that are not a part of dictionary word.
We just want to capture those rows which has more than five consonants
appearing in a string continuously.
Then, we will manually review whether it is mistyped or not.
I Hope I am able explain my question.

Re: How to identify whether the value of column has 5 or more consonants without vowel?

Reeza — Wed, 11 Apr 2018 18:37:59 GMT

So a mistyped word will not end up in the list and then you can check it...am I missing something?

What defines a 'mistyped' word is your actual problem.

Re: How to identify whether the value of column has 5 or more consonants without vowel?

nikunjgattani — Thu, 12 Apr 2018 01:39:37 GMT

We have to find a mistyped word.
So, to capture mistyped word we came up with a logic that any word which
doesn't have vowels in it could be mistyped.
On the top of that we added that if the count of consonants(continuous)
should be greater than 5 or more then only we call them mistyped.

*This column contains addresses.

Re: How to identify whether the value of column has 5 or more consonants without vowel?

nikunjgattani — Fri, 13 Apr 2018 14:58:22 GMT

Hi All,

Below is the solution I got from a person in linkedin -

data youdata;
infile cards truncover;
input Column_Test $ 200.;
cards;
I have a string column named Column_Test which cotains text up to 200 characters
I have a string column named Column_Test which cotains text up to 2000000 as
I have a string column named Column_Test which cotains text up to 200 dfgt
I have a string column named Column_Test which cotains text up to 200 dfgty
;
run;

data mistyping;
length id 8. ;
set youdata;
id = _N_;
keep id;
if prxmatch('/[^aeiouAEIOU\s]{5,}/', Column_Test);
run;

Thank You all for trying to solve my problem.

Re: How to identify whether the value of column has 5 or more consonants without vowel?

nikunjgattani — Mon, 16 Apr 2018 14:57:29 GMT

Yes, you were correct. 'Y' should also be included as vowel.

I have included 'Y' in my solution. I have posted the solution too in one of the comment.