So long, so expensive!   The drug approval process.   There is an old joke in the pharmaceutical industry.  I paraphrase:   What’s a drug’s favorite game? "Hide and Seek"—because it takes years to find its way to the pharmacy!   There are many minor daily mishaps such as missing your bus or spilling coffee on yourself before that important meeting which can feel frustrating but often don't have lasting consequences.   The detrimental unintended consequences of a new drug’s effect on humanity will be the opposite of a minor daily mishap; this will be life or death.   That is why the drug approval process has so much rigor built into it.   Let’s explore!   Figure 1 Drug approval process - Follow Link 1. in Supporting material below   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   The Data and Analysis pieces are what we are interested to optimize using SAS. These are in order:   The Investigation New Drug (IND) process:     Investigational New Drug (IND) application is submitted to regulatory agency based on the results from initial testing to determine whether drug/biological product can be tested on humans.   SAS can make this quicker by helping create:   Single version of data truth and workflow, permissions and regulatory reporting and management (SAS special talent) Explanatory Analytics (SAS special talent) Exploratory Analytics (SAS special talent) Predictive and future based analytics (SAS special talent) Optimization analytics (SAS special talent) Visualization, Dissemination and workflow of data single version of the truth and all analytics (SAS special talent).   Phase 1 - First In Human Study (FIH)     This is the first-in-human (FIH) study. It focuses on a drug’s side effects, optimal dosage ranges, and how it is metabolized in the body.​   SAS can make this quicker by helping create:   Single version of data truth and workflow, permissions and regulatory reporting and management (SAS special talent) Explanatory Analytics (SAS special talent) Exploratory Analytics (SAS special talent) Predictive and future based analytics (SAS special talent) Optimization analytics (SAS special talent) Visualization, Dissemination and workflow of data single version of the truth and all analytics (SAS special talent).   Phase 2 - Validity testing.  Retesting results from Phase 1 for validity     Preliminary data is obtained on whether the drug works in people with a certain disease or condition. Safety and short-term side effects are evaluated.​   SAS can make this quicker by helping create:   Single version of data truth and workflow, permissions and regulatory reporting and management (SAS special talent) Explanatory Analytics (SAS special talent) Exploratory Analytics (SAS special talent) Predictive and future based analytics (SAS special talent) Optimization analytics (SAS special talent) Visualization, Dissemination and workflow of data single version of the truth and all analytics (SAS special talent).   Phase 3 and pre NDA meeting - More Data, more complex testing, more NDA Compliance     Phase 3 is a larger-scale phase that gathers additional regarding the safety and efficacy of the drug and studies different populations. Furthermore, regulatory agency gathers in a Pre-NDA/BLA meeting to analyze results, combined and compared with the previous phases. This is where the single version of the truth comes into its own.   SAS can make this quicker by helping create:   Single version of data truth and workflow, permissions and regulatory reporting and management (SAS special talent) Explanatory Analytics (SAS special talent) Exploratory Analytics (SAS special talent) Predictive and future based analytics (SAS special talent) Optimization analytics (SAS special talent) Visualization, Dissemination and workflow of data single version of the truth and all analytics (SAS special talent).   The NDA Submission - And all of that culminates into the New Drug Application (NDA) to the regulatory agency(s).     New Drug Application (NDA) is submitted to the regulatory agency to seek approval to market the drug.​   The drug sponsor submits the NDA, or the New Drug Application, to the regulatory agency to formally ask for the agency’s approval to market the new drug.   The NDA includes all the animal and human data collection and analyses as well as information about how the drug behaves in the human body and how it is manufactured. ​ ​   It is important to note here that as a SAS Clinical Programmer, you are responsible for gathering all the data from phases 1, 2, and 3 and generating all the results that are to be submitted as a part of the NDA.   This requires submission of all the results from the 3 phases.   This is why you need SAS; all you have done up to this point leads you to a submission.   If you do this with SAS, the single data version allows you to re-use and build on previous phases, so things happen quickly.   The analysis and preparation allow you to cut analytical mistakes, to show a yellow brick road of metadata that led you to all conclusions, and the regulating agencies accept SAS as a way to look at results already, so you don’t need a lot of reformatting to submit.   Phase 4 - And just when you thought you were done.....  Long term study     Since it is not possible to predict all of a drug’s effects during clinical trials, safety issues after the drug goes on the market are monitored during this phase. The sponsor, typically the manufacturer, is required to submit periodic safety updates to the regulatory agency. The role of the agency’s post-marketing safety system is to detect serious unexpected adverse events and take definitive action when needed.   One system through all the stages.   The detrimental unintended consequences of a new drug’s effect on humanity will be the opposite of a minor daily mishap; this will be life or death.   Using rigorous data management practices and rigorous analytical processes and rigorous regulatory compliance preparation, you can err on the side of life, in a quicker and cheaper way.   SAS Life Science Analytics framework.   Supporting Material   https://conquer-magazine.com/issues/special-issues/january-2021-a-patient-guide-to-recent-fda-approved-oncology-drugs/2021-fda-drug-approval-process-infographic            Find more articles from SAS Global Enablement and Learning here. ... View more

So what did we do?   Researchers often encounter difficulties in comprehending and integrating information from disparate protocols, hindering the progress of scientific inquiry.   This use case is aimed at creating a Unified Studies Definition Model (USDM) by translating diverse human-readable protocols into a standardized format.   The envisaged solution combines the versatility of Excel spreadsheets and the efficiency of Natural Language Processing (NLP) techniques to bridge the gap between heterogeneous study designs.   The primary objective of this use case is to   enhance the clarity, accessibility, and interoperability of research protocols   This should lead to development of a standardized, machine-readable representation USDM.   The following steps are used in the use case:   Used contextual information extraction from a couple of clinical trial protocols (early and late phase) and stored the relevant info into the USDM (excel) workbook ... in the right place (in the right field). Key steps included contextual information extraction from clinical trial protocols and ensuring scalability to handle various protocols. Employing SAS Natural Language Processing (NLP) techniques and Large Language Models (LLM's); automatically translate human-readable study protocols into a structured format. This involves extracting key information such as study objectives, methodologies, inclusion/exclusion criteria, and outcomes. A robust approach has been developed, combining natural language processing (NLP), text analytics and a large language model (LLM) to handle clinical trial protocols.  Whenever LLMs are involved, it's essential to discuss cost, security, and privacy. However, as a life sciences advisor, the primary concern is: "How can I trust a generative LLM to deliver reliable, non-hallucinated results? How can I establish guardrails to ensure this? How do I address the security and privacy of my data?"    Evaluating Gen AI and Text Analytics for creating a USDM   Some initial results after pre processing, highlights the extractions of inclusion and exclusion criteria by LITI rules and LLMs.      Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.     There are several challenges to overcome, such as overfitting in LITI rules and the need for extensive pre-processing.   The use of various Python packages for splitting protocols into chunks and extracting tables and images were investigated.    Preliminary Outcomes: The use of LLMs like LLAMA-V2 and RAG-for-LLMs proved beneficial, though resource-intensive. Leveraging LITI rules as pre-filters, for confidence scoring, to reduce costs by pre-filtering data and thereby leading to quicker and accurate results seems beneficial.    A bit more on LITI rules for confidence scoring: Confidence Scoring in SAS Viya VTA – LITI    The same corpus used for information extraction with the LLM is scored against the LITI rule to verify the presence of the information.   A confidence score, ranging from 0 to 1, is calculated based on the number of relevant terms matched by the LITI rule.   Higher scores indicate better extraction quality, while lower scores suggest hallucinations or inaccuracies.   This method provides a robust metric to evaluate different LLM models or prompts.  (5 steps to improve information extraction using trustworthy generative AI - The SAS Data Science Blog)    Benefits of integrating SAS NLP and Gen AI for creating a USDM   Avoiding hallucinations: The NLP and text analytics pre-filtering process assimilates the most relevant source data from various documents, ensuring the outputs are more accurate and reliable.  Enhancing time to value: By pre-filtering the data, a smaller LLM can handle GenAI tasks more efficiently, leading to quicker results. Providing more focused context to an LLM significantly enhances output quality, especially for weaker models.  Ensuring privacy and security: Using a local vector database for fine-tuning generative models is possible. This gives users only relevant embeddings to the LLMs via APIs or localized instances of the LLM, ensuring the privacy and security of sensitive data.  Reducing costs: Text analytics and NLP significantly reduce the amount of information sent to the LLMs. In some cases, only 1 – 5% of the overall data is used for answers. This eliminates the need for excessive external API calls and reduces the computational resources required for localized LLMs. Processing documents with GPT-4 involves significant costs. For documents ranging from 3,500 to 8,000 tokens, processing 1,000 documents costs between $105 and $240 for input alone. The output cost is often 2-3 times higher per token than the input. With GPT-4 pricing at $30 per million tokens, these costs can add up quickly.  Traceability: SAS® Viya® enables end-to-end verification and traceability of results, helping users to verify information and trace it back to the statements from which the summaries were derived, potentially thousands of statements. This traceability feature enhances transparency and trust in the generated outputs.  Establishing guard rails: establish guard rails to control the information that’s sent to the LLMs and also analyse the output – quality checks​. Adopting a “trust but verify” approach ensures that LLMs’ extractions, which can impact downstream tasks, are checked and validated to prevent unchecked errors.    So what? - A few open ended conclusions   The work highlights the potential of advanced data management techniques in transforming clinical trial protocols.  By embracing innovative technologies and collaborative efforts, the industry can achieve greater efficiency and accuracy in clinical research.  These examples merely hint at the vast potential unlocked when combining the precision of linguistic methods in SAS NLP with LLMs. These techniques not only tackle quality issues in text data but also integrate subject matter expertise, granting organizations significant control over their corpora. In some instances, the corpus size for fine-tuning can be reduced by up to 90%. By curating higher-quality data for fine-tuning, you can achieve more accurate responses from LLMs, reduce the occurrence of hallucinations, and establish a method for validating responses.    References:   *Presented at the 2024 Europe CDISC+TMF Interchange and SAS Innovate 2024, Jasmine Kestemont from Innovion and Stijn Rogiers from argenx presented their groundbreaking work on translating human-readable protocols into machine-readable formats using the Unified Study Data Model (USDM).    *SAS Generative AI explained (GenAI)     Find more articles from SAS Global Enablement and Learning here. ... View more

Why Trust?   The one issue that persist with any manipulated data output is trust.    In an interesting article I read recently, published by Caltech Science Exchange, the question of Trust came in in context of AI. (Can we trust artificial intelligence)   According to this author, one of the big impactful events influencing trust is Data and amongst other issues, the Bias it contains.  And their fix, make sure every data set used to answer any question is scrutinized to determine its ‘motivation, composition, collection process, recommended uses and so on’ So What?   Essentially, moving back into the safety of single version of the data truth will show us the picture of whence our intelligence is derived. Results are easily manipulated if there is no single version of data for everyone. Results can be skewed where there is no oversight, no process, no checks and balances to ensure auditability and traceability. In other words, if you want trust, we need to build a world where even under data torture and/or duress, this singular data version will provide repeatable, reliable ‘truths’ if the same questions are asked by whoever.   Enter SAS Clinical Acceleration Repository (CAR).   SAS CAR lets you easily integrate large and diverse data sets such as electronic data capture (EDC) systems, in-house clinical data management systems (CDMS), labs and contract research organizations (CROs).   The CAR Solution from SAS is helping you trust your version of the truth. Some of its primary features includes but are not limited to: Centralized global repository. It consolidates clinical information into a single, secure, centralized global repository Data Tracing. It traces data pedigree back to the origin. Data Security. Defines data check-in/check-out processes. Maintains audit trails (see number 4 below). Provide you with Electronic Signatures, so the individuals that touch data must make themselves known. Supports versioning; and allows for role-based privileges (I’ll show how easy this is to set up a bit later). Audit Trails. Let’s you readily determine what audit changes were made, when and by whom for all the content stored in the repository. Secure Logins. Controls all information and research team access via secure logins. Regulatory Compliance. Enables compliance with the FDA’s Title 21 CFT part 11 requirements, as well as other industry regulations. CDISC compliance. Complies with CDISC and its initiatives – dataset-JSON and CDISC CORE. Medical Device Reporting. Supports integration with MDR. Cloud Native too?   Yes, SAS Clinical Acceleration Repository (CAR) is open and cloud native. Lets Wrap SAS CAR up. With all said and done, if you remember only three things about SAS CAR it is this:   Data Integrity   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.   Open Repository   Easy to use   Next Steps Come and ask for a Demo, give us a try, this may be the game changer you have been seeking.   SAS Clinical Acceleration Main Page: https://www.sas.com/en_us/software/clinical-acceleration.html   Request a Demo: https://www.sas.com/en_us/software/how-to-buy/request-price-quote.html   ... View more

  Is that a good site?   This is a question most frequently asked when designing an Enrollment plan for clinical trials, and determining the answer without analytics to help may be a daunting task for sure.   Before we dive too deeply into the solution, lets figure out what we are trying to solve for.   Clinical trials ultimately aim to answer two simple questions: Is the drug or device safe? and Does it do what it’s supposed to do? This can take decades and cost astronomical amounts…take the case of Tamoxifen for Breast Cancer   A clinical trial started in January of 1994 and is projected to complete only in September of 2024 being conducted by the UK’s Queen Mary University of London.   The trial enrolled an astonishing 7,154 women aged 35-70 years across nine countries to investigate if tamoxifen, a hormonal drug, can prevent the development of breast cancer in high-risk women.   And herein lies the opportunity.   Question: How can I accelerate the speed and cut the cost of Clinical Trials.   Answer: Simply by using the analytics that is embedded in the SAS® Clinical Enrollment Simulation product to create your Enrollment Plans for your clinical trial.   But it will be daunting and complex to build an executable enrollment strategy.  The factors used as input are myriad and complex.   The graphic below details the factors that is typically considered when building a representative and executable Enrollment Plan.   Select any image to see a larger version. Mobile users: To view the images, select the "Full" version at the bottom of the page.         These factors are the things that, when manipulated and fine tuned just the right amount, ensure that your Enrollment Plan is on budget and in time.   Why is manipulation of the input variables important?   Some estimations from Statista.com put the cost per clinical trial participant at upwards of $30,000.  Dependent on the clinical trial that is.  The graphic below shows some clinical trial costs running into the multiple hundreds of thousands dollars.  Phew!.   That means that if we take the lowest cost of $30,000 per patient, the Tamoxifen Breast Cancer study mentioned in the opening has cost 7,154 times $30,000, so $215 odd hundred million so far.  That is not a number, it is a SOUND... something like aaaaaaaghhhhhh!.           Two approaches to manipulating the input variables   There are two approaches ways to manipulate the factors that drives the success of a Enrollment Plan. Hint: only one of them is flexible and simple. Projecting Enrollment approach (a.k.a - Guesswork based on previous data) Simulate Enrollment approach (a.k.a – Get statistics to optimize the best potential success) Projecting Enrollment Approach       Key challenges from the traditional approach:   Working backwards form the goal is really trying to make a scenario that can justify the goal we are trying to hit. It does not always have a chance of actual execution and success. It uses actual value inputs. No room for variability.  Let’s say for instance group A will be ready in 5 months but really, we know it will be sometime between 4.5 and 7.  That variation is not able to be considered. Enrollment decay is not considered. What typically happens: Dr tells you she can plan on enrolling 15 patients per month.  You know that is optimistic, so you enter an actual value i.e., 12 to be safe. Again, there is not accounting for actual decay as the low figure is hard coded for the entire time. Never get anyone enrolled. This is probably an underestimate, and closer to 30% never enroll anyone.  The coordinator leaves the clinic, so no one runs the trial. A competing trial starts up and they send their patients to that one.  The site just has delays in paperwork and just gives up It can take a long time to create and compare multiple scenarios to determine the impact of changing country and/or site profiles using Excel, so Cross regional data and costs typically are siloed. This approach provides no measure of risk – for completing the trial on time or on budget. What-if scenarios. No insight or corrective measures regarding the actions needed to keep a trial on track. Simulate Enrollment Approach   SAS® Clinical Enrollment Simulation can accurately simulate and model the complex variabilities and probabilities associated with enrolling subjects in clinical trials, including country and site selection using analytics.   The three main benefits of using the SAS solution for running your Clinical Enrollment are detailed in the three graphics below.       The SAS Enrollment Simulator allow you to calculate the chance of success - what I really want to know is “am I going to hit my targets.” Simulation allows you specify what targets matter, then we can look at what we know about the sites and see how likely we are to achieve the study goals.         The SAS Enrollment Simulator allow you to compare various scenarios with the Milestone achievement capability. This allows us to see ‘what happens if’.  We can play with different scenarios to give us the biggest likelihood of achieving success in our clinical trial.       The SAS Enrollment Simulator allow you to intervene at any time and play out scenarios for intervention when you need to ‘rescue’ a clinical trial site.   For instance, in this scenario we can see that by adding two extra ‘rescue sites’ bumps my chances of getting back on target to 75% and adding three to 95%.   This allow me to make a business decision to see whether spending the money will help me be successful or not, and by what percentage point.   To learn more Education: SAS Clinical Enrollment Simulation education course Help Files: SAS Clinical Enrollment Simulation help files  Support: SAS Clinical Enrollment Simulation Documentation and Support Product Page: SAS Clinical Enrollment Simulation Product Page   ... View more