Hey Ksharp,   Thanks for this - this code is a bit too complex for me but I'll see if our statistician can get this running.    Many thanks,   Craig  ... View more

Hi Ksharp,   I see  - thank you. Would it be possible for you to write an example with it please? Not quite sure how to do it.    Many thanks,   C     ... View more

Hi Paige,    Sorry was on a course w limited access to device.   Of course,   here is the  corresponding log    1643 1644 PROC LOGISTIC DATA=BIO3 PLOTS = ROC PLOTS(MAXPOINTS=NONE) 1645 plots=calibration(CLM ShowObs); 1646 MODEL HNC(EVENT='HNC') = PROB / ROCCI GOF OUTROC = ROC; 1647 RUN; NOTE: PROC LOGISTIC is modeling the probability that HNC='HNC'. NOTE: Convergence criterion (GCONV=1E-8) satisfied. WARNING: This graph has too many graphical elements. You may not be able to get any vector graphics output and in that case, you can set your output format to an image type. WARNING: This graph has too many graphical elements. You may not be able to get any vector graphics output and in that case, you can set your output format to an image type. NOTE: The number of observations of the LOESS plot (384616) exceeds the limit of 5000. Specify the LOESSMAXOBS option of the ODS GRAPHICS statement to override the limit. NOTE: The number of observations of the LOESS plot (384616) exceeds the limit of 5000. Specify the LOESSMAXOBS option of the ODS GRAPHICS statement to override the limit. NOTE: There were 384616 observations read from the data set WORK.BIO3. NOTE: The data set WORK.ROC has 77689 observations and 8 variables. NOTE: PROCEDURE LOGISTIC used (Total process time): real time 4:43.01 cpu time 3:42.45   ... View more

Hey Ballard,   Thanks for the help - that did it.   It did run for an hour and a half but lacked memory due to the size. May need to consult with colleuages r.e a more powerful computer.    Cheers    🙂 ... View more

Sorry for the delay in getting back - swamped w things.    So I have attempted to make sure all variables are the same in nature and format. Only exception is outcome variable but if this is binary guessing it doesn't matter?   I have exported the model coefficients from the development file into a .rtf file. - is this correct or what would be the next steps?   Many thanks,   Craig    FILENAME COEF 'J:\MED\DentalSchool\DPHU\CraigSmith\ARCAGE\model_coef.txt.'; /* Step 2: Load the coefficients from the file */ data COEF; infile 'model_coef.txt'; input VARIABLE : $20. ESTIMATE; run; /* Step 3: Merge the coefficients with the validation dataset */ proc sql; create table VALID_COEF as select A.*, B.ESTIMATE from VALID A left join COEF B on A.VARIABLE = B.VARIABLE; quit; /* Step 4: Calculate the predicted probabilities */ data VALID_PRED; set VALID_COEF; PROB = 1 / (1 + exp(-(& 0.4692. + ESTIMATE))); run; /* Step 5: Calculate the model performance */ proc logistic data=VALID_PRED; model HNC (event='1') = PROB / nofit; roc 'Model Performance' / ci=none; run;         ... View more

Thanks Dave    I'll try this tomorrow and report back 🙂 ... View more

Forgive me, I am struggling to follow this.    So I need to validate the model using the stacked data, assess ROC with confidence intervals and assess calibration. The validation needs to use the same coefficients from the original dataset. The variables and reference values shouldn't change but how would I rewrite it so these things exist for HPLogistic please? Could you even modify the statement below to show me what needs changed please?    Many thanks,    Craig    *PROC LOGISTIC DATA = BOTH plots=calibration(CLM ShowObs); FORMAT SEX SEX. EDUCATC EDUCATC. SMOKE SMOKE. DRINK DRINK. ALCOIC ALCOIC. ; CLASS SEX ( REF = '0: Female' ) EDUCATC ( REF = '16+ Years' ) SMOKE ( REF = '0: Never') DRINK (REF = '0: Never') ALCOIC (REF = '6: Never') ; MODEL HNC = AGE SEX EDUCATC SMOKE SMOKP DRINK ALCOIC / ROCCI ROCCONTRAST RL GOF; RUN;     ... View more

Hi Stats Dave,   Thanks for your advice - I've drawn up some code using the links your provided. Can I just check this code with yourself - variable names should match and I'm looking to have ROC values with CIs /  plot and calibration plot ?    Best wishes,   Craig    *XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX !! TEST CODE FOR VALIDATION !! N.B ORDER OF CASE AND CONTROL?? XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX; *PROC HPLOGISTIC DATA = BOTH DESCENDING plots=calibration(CLM ShowObs); FORMAT SEX SEX. EDUCATC EDUCATC. SMOKE SMOKE. DRINK DRINK. ALCOIC ALCOIC. ; CLASS SEX ( REF = '0: Female' ) EDUCATC ( REF = '16+ Years' ) SMOKE ( REF = '0: Never') DRINK (REF = '0: Never') ALCOIC (REF = '6: Never') ; MODEL HNC = AGE SEX EDUCATC SMOKE SMOKP DRINK ALCOIC / ROCCI ROCCONTRAST RL GOF; RUN; ... View more

Hey StatsDave,   Thanks for the answer. Ah I see - thank you for the explanation. I was informed that coefficients i.e the Linear Predictor needed to be the same for validation - I'm assuming this method does this and fits TRIPOD guidelines?   I'll draw up some code and can try with a colleague next week and report back.    Many thanks,   Craig   ... View more

Het Stat Dave,   Thank you for your explanation - that resource for discrimination / calibration is helpful - I have used several of these during model development 🙂 So to clarify - do I "merge" both datasets? Or what is the process for this? I have the coefficients from the development model but am unsure how to proceed from there.    Have a nice Easter,   Craig  ... View more

Got it workign was missing quotes - thanks so much Paige !!   One thing though - how can I get an entry of data to belong to 2 different groups simultaneously. As you'll see, HNC is all of the sites (and groups) combined.     Best wishes,   Craig  ... View more

Hey Ballard,   Thanks for the message.   I see - i replaced it with the name of my dataset but that still hasn't solved it.    KR,   Craig   569 DATA HNC; 570 SET HNC; 571 IF SUBSITE IN = (C00.3, C00.4, C00.5, C00.6, C00.8, C00.9, - 390 76 ERROR 390-185: Expecting an relational or arithmetic operator. ERROR 76-322: Syntax error, statement will be ignored. 572 C02.0, C02.1, C02.2, C02.3, C02.8, C02.9 573 C03.0, C03.1, C03.9 574 C04.0, C04.1, C04.8, C04.9 575 C05.0 C05.1, C05.2, C05.8, C05.9 576 C06.0, C06.1, C06.2 C06.8, C06.9) 577 THEN SUB = 'OCC'; 578 579 IF SUBSITE IN = (C01.X, - 22 ----- 557 ERROR: The right-hand operand must be an array name or a constant value list. The specified name NAME, is not an array. ERROR: DATA STEP Component Object failure. Aborted during the COMPILATION phase. ERROR 22-322: Expecting a name. ERROR 557-185: Variable C01 is not an object. NOTE: The SAS System stopped processing this step because of errors. NOTE: DATA statement used (Total process time): real time 0.00 seconds cpu time 0.00 seconds 580 C02.4, 581 C09.0, C09.1, C09.8, C09.9, 582 C10.0, C10.1, C10.2, C10.3, C10.4, C10.8, C10.9 583 C14.0, C14.2) 584 THEN SUB = 'OPC'; 585 586 IF SUBSITE IN = (C32.0, C32.1, C32.2, C32.3, C32.8, C32.9) 587 THEN SUB = 'Larynx'; 588 589 IF SUBSITE IN = (C00.3, C00.4, C00.5, C00.6, C00.8, C00.9 590 C01.X, 591 C02.0, C02.1, C02.2, C02.3, C02.4, C02.8, C02.9 592 C03.0, C03.1, C03.9 593 C04.0, C04.1, C04.8, C04.9 594 C05.0, C05.1, C05.2, C05.8, C05.9 595 C06.0, C06.1, C06.2, C06.8, C06.9 596 C09.0, C09.1, C09.8, C09.9 597 C10.0, C10.1, C10.2, C10.3, C10.4, C10.8, C10.9 598 C11.0, C11.1, C11.2, C11.3, C11.8, C11.9 599 C12.X, 600 C13.0, C13.1, C13.2, C13.8, C13.9 601 C14.0, C14.2, C14.8 602 C32.0, C32.1, C32.2, C32.3, C32.8, C32.9) 603 THEN SUB = 'HNC'; 604 605 606 RUN; ... View more