BookmarkSubscribeRSS Feed
mar00390
Fluorite | Level 6

If I want to find possible and probable cases of a disease using full-text search from a cohort of people, what would some suggestions be?

 

For example, I have two variables: PatientNumber and Text. In some cases, the text tells me immediately - "Possible diagnosis" - but other times, the text never uses "possible" or "probable." I don't know if I can search for "might have diagnosis" because I don't think doctors write that, but I don't want to eliminate possible cases just because they don't have the word "possible" or "probable" in it.

 

Any ideas would be useful. Please help!

2 REPLIES 2
mkeintz
PROC Star

Take a sample of the unclassified text (i.e. that don't have "probable" or "possible").  BTW, you did screen for negatives right, like "not probable"?.  See if there are expressions you conclude mean probable or possible.  Add those expressions to your classification logic.  Apply to your data, thereby reducing the number of unclassified.

 

Repeat the above with a sample of the remaining unclassified text.

--------------------------
The hash OUTPUT method will overwrite a SAS data set, but not append. That can be costly. Consider voting for Add a HASH object method which would append a hash object to an existing SAS data set

Would enabling PROC SORT to simultaneously output multiple datasets be useful? Then vote for
Allow PROC SORT to output multiple datasets

--------------------------
Reeza
Super User
Find all of the words that are diagnosis in your file.

Find all words on either side, accounting for punctuation, using a two to five word window. Look at that data to determine your rules.

hackathon24-white-horiz.png

The 2025 SAS Hackathon has begun!

It's finally time to hack! Remember to visit the SAS Hacker's Hub regularly for news and updates.

Latest Updates

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

SAS Training: Just a Click Away

 Ready to level-up your skills? Choose your own adventure.

Browse our catalog!

Discussion stats
  • 2 replies
  • 699 views
  • 3 likes
  • 3 in conversation