BookmarkSubscribeRSS Feed
campbell_ms
Calcite | Level 5

I work in health care and I've built this big model to measure physician performance on a variety of things like clinical care, patient satisfaction, clinical integration, and cost and I've written in all in SAS.  It pulls from a variety of tables in a data warehouse but also some external sources like spreadsheets and datasets.  It gathers massive amounts of data, rearranges it, computes a lot of variables, and finally calculates an index…a physician's score based on the scores of all other physicians.  It creates a final table of calculations and 2 ODS scatter plots output into a PDF.

Now, it's time to implement and given time and resource constraints the production team has decided to implement my SAS code into production instead of translating it into Informatica (their standard).  What I'm dealing with now is the QA question…how do we put this through QA?  One, they don't know how to program in SAS.  Two, they don't have the capability of understanding the mathematical methods within the SAS code.  Three, their typical QA MO is to QA by example; take an observation, follow it through the process, and make sure it comes out the other side.  That's not possible in this model because of the cumulative nature of the calculations. 

So what I'm trying to do is come up with some recommendations for how a QA team that doesn't write SAS code can QA a SAS code. 

What I have already is model validation…steps I went through to ensure I was identifying the right sources, pulling the right variables from those sources, manipulating the data correctly in theory, making the right assumptions, and applying the right mathematical methods.  This was mostly peer review.  They're concerned with the implementation of the SAS code itself.

Got any experience with this?  Got any ideas?

Melissa

1 REPLY 1
Doc_Duke
Rhodochrosite | Level 12

Melissa,

Another way to address "QA by Example" in the statistics world is 'regression testing'.  Rather than following a single observation through the process, use the same set of inputs for both the development and the production systems and see if you get the same results. 

[The validation suite that SAS used to send with V8 would do just that.  It would send known data through the procedures and run PROC COMPARE on the results.]

One, more general, concern I have about doing this scoring with production systems is data completeness and having the tools and processes in place to know the degree of completeness to help interpert the results.

Doc Muhlbaier

Duke

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

New Learning Events in April

 

Join us for two new fee-based courses: Administrative Healthcare Data and SAS via Live Web Monday-Thursday, April 24-27 from 1:00 to 4:30 PM ET each day. And Administrative Healthcare Data and SAS: Hands-On Programming Workshop via Live Web on Friday, April 28 from 9:00 AM to 5:00 PM ET.

LEARN MORE

Discussion stats
  • 1 reply
  • 920 views
  • 0 likes
  • 2 in conversation