topic Re: Flagging records following exclusion criteria in a specific part of report in SAS Programming

Flagging records following exclusion criteria in a specific part of report

DeepakSwain — Thu, 12 May 2016 16:44:38 GMT

Hi there,

I am trying to create flag for a report having multiple sections by applying exclusion criteria. As the exclusion criteria are also existing in other part of the same report, the outcome for each section is also affected. At the same time, I also want to execute my query at specific section of the report. Can you kindly guide me further. I have given below a very simple sas just to begin the flow of discussion (the worst code you might have ever seen).

DATA TEST;
REPORT= "1) POLYP AT LEFT COLON: TUBULOVILLOUS. NEGATIVE FOR SERRATED.
2) POLYP AT RIGHT COLON: INFLAMMATORY. NEGATIVE FOR TUBULOVILLOUS. 
3) POLYP FROM TRANSVERSE COLON: NEGATIVE FOR INFLAMMATORY BUT SERRATED IS FOUND.";

/*INCLUSION CRITERIA*/
IF FIND(REPORT, TUBULOVILLOUS)  THEN TUBULOVILLOUS=1;
IF FIND(REPORT, INFLAMMATORY)  THEN INFLAMMATORY =1;
IF FIND(REPORT, SERRATED)  THEN SERRATED =1;      

/*EXCLUSION CRITERIA*/
IF FIND(REPORT, "NEGATIVE FOR TUBULOVILLOUS") THEN EXCLUDE_TUBULOVILLOUS=1;
IF FIND(REPORT, "NEGATIVE FOR INFLAMMATORY") THEN EXCLUDE_INFLAMMATORY =1;
IF FIND(REPORT, "NEGATIVE FOR SERRATED") THEN EXCLUDE_SERRATED =1; 

/*FINAL FLAGGING*/
IF TUBULOVILLOUS=1 AND EXCLUDE_TUBULOVILLOUS NE 1 THEN FLAG_TUBULOVILLOUS=1;
IF INFLAMMATORY =1 AND EXCLUDE_INFLAMMATORY NE 1 THEN FLAG_INFLAMMATORY =1;
IF SERRATED =1 AND EXCLUDE_SERRATED NE 1 THEN FLAG_SERRATED =1;
RUN;

Thanks in advance for your kind reply.

Re: Flagging records following exclusion criteria in a specific part of report

ballardw — Thu, 12 May 2016 18:59:11 GMT

Not even close to worse I've seen (or written).

The only thing I see at this time is the first inclusion criteria is missing quotes for the literal string comparison values as were correctly used in the exclusion. Other than that it looks like the code should do what I think you are attempting.

If you have not UPCASEd the Report string prior to this or use the FIND modifiers i and t in the comparisons.

Re: Flagging records following exclusion criteria in a specific part of report

DeepakSwain — Thu, 12 May 2016 20:02:36 GMT

Hi there,

While flagging report part 1 due to presence of inclusion criteria, it is not flagged because its' exclusion criteria is avalable in report part 2. So i am looking for some code which will restrict my exclusion criteria to specific part of the report and it will not see the complete report for exclusion.

Regards,

Deepak

Re: Flagging records following exclusion criteria in a specific part of report

ballardw — Thu, 12 May 2016 20:57:38 GMT

You will need to clarify what "exclusion criteria is avalable in report part 2" means. Are you talking about a different data set? Are you trying to match some flags set in a different data set to this one? Are you want to use variables that are not actually in your current data set?

You may need to provide a much more complete example of what you have and what the expected result should be. Please post any data as data steps.

Re: Flagging records following exclusion criteria in a specific part of report

DeepakSwain — Fri, 13 May 2016 12:57:34 GMT

Hi Ballardw,

For your kind information, each report of a patient is having multiple sections such as 1) 2) 3). All three sections are in a single record. When I try to flag identifying specific words as well as exclusion criteria, because of the existence of same exclusion word in other section of the report, I could not flag it.

I have attached one excel sheet explaining what I am getting and what is my expectation.

Thank you in advance for your kind reply.

Regards,

Deepak

Re: Flagging records following exclusion criteria in a specific part of report

ballardw — Fri, 13 May 2016 15:36:45 GMT

You have these requirements for "final flagging"

IF TUBULOVILLOUS=1

IF INFLAMMATORY =1

IF SERRATED =1

BUT in your example code they are all missing so the final flags cannot be set.

Also you have value that set EXCLUDE_TUBULOVILLOUS, EXCLUDE_INFLAMMATORY and EXCLUDE_SERRATED to 1

so they would fail the "NE 1" and not set those flags. If you expecte to set those last 3 flags to 1 then you need to reconsider the logic.

I think that you need

IF FIND(REPORT, "TUBULOVILLOUS") THEN TUBULOVILLOUS=1;

IF FIND(REPORT, "INFLAMMATORY") THEN INFLAMMATORY =1;

IF FIND(REPORT, "SERRATED") THEN SERRATED =1;

so those 3 variables are not missing.

You need more logic if you are trying to find the presence of the phrase

NEGATIVE FOR INFLAMMATORY in a different location from simple INFLAMATORY. Such as actually using the result fo FIND as a position and compare results.

And you will need to account for the key word appearing alone before the Negative and after the negative (case with serrated), which FIND alone isn't as well suited for as it reports on the first found location, so you need to do a lot more parsing on your strings.

First: determine the maximumn number of times any of your key words may appear, then determine a comparsion between the locations and the eqivalent number of "Negative for" to see if the key word is occuring alone or in combination

Re: Flagging records following exclusion criteria in a specific part of report

DeepakSwain — Fri, 13 May 2016 16:04:31 GMT

Hi Ballardw,

Can you kindly suggest some logic which will help me to flag part "1)" of the report as FLAG_TUBULOVILLOUS=1 ignoring the EXCLUDE_TUBULOVILLOUS =1 as the exclusion is existing in part "2)" of the same record and not in part "1)".

Thank you in advance for your kind reply.

Regards,

Deepak

Re: Flagging records following exclusion criteria in a specific part of report

ballardw — Fri, 13 May 2016 17:35:49 GMT

@DeepakSwain wrote:

Hi Ballardw,

Can you kindly suggest some logic which will help me to flag part "1)" of the report as FLAG_TUBULOVILLOUS=1 ignoring the EXCLUDE_TUBULOVILLOUS =1 as the exclusion is existing in part "2)" of the same record and not in part "1)".

Thank you in advance for your kind reply.

Regards,

Deepak

Here is a brief example of using the optional Start position for Find. The line with Example2 starts searching at the position that the "2)" is found.

data example;
   report ="1) THIS IS JUST AN EXAMPLE 2) TO SEARCH FOR EXAMPLE AFTER THE 2)";
   Example1 = find(report,'EXAMPLE');
   Example2 = find(report,'EXAMPLE',find(report,'2)'));
run;

Re: Flagging records following exclusion criteria in a specific part of report

DeepakSwain — Fri, 13 May 2016 20:08:06 GMT

Hi Ballardw,

This is a nice lesson you have given me in context of use of find function nested within another find function. Definitely I am going to use it in my future analysis.

Unfortunately in this case I don't know the delimiter to be 2. or 2) or B. or B); so I will be looking for some other alternative approach such as use of perl. or some loop.

Once again thank you for a wonderful tips of using double find function.

Have a great weekend.

Regards,

Deepak