cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
whs278
‎11-20-2023
  • 64 Posts
  • 46 Likes Given
  • 1 Solutions
  • 6 Likes Received

Re: %PUT STATEMENT WONT PRINT TO LOG

by Quartz | Level 8 in SAS Programming
‎08-04-2021 12:16 PM
1 Like
‎08-04-2021 12:16 PM
1 Like
  Unfortunately, there is a lot of stuff before for that code so it's hard to pinpoint /*---------------------------------------------------------------- /* ORGANIZATION /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - /* /* FIRST CREATED ON: 03/28/2018 /* CREATOR: INITIALS /* /* CODE BORROWED FROM: /* /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - /* PROJECT: CTE /* /* /* PROGRAM FUNCTION: CREATE HSAP APPLICATION FILE /* WITH CTE PROGRAM INFORMATION /* CREATE ANALYSIS VARIABLES /* ADD CTE COURSE DETAIL VARIABLES /* /* /* DATA IN: /data/production/data/Master/HSAP_APPS20072018 /* /data/projects/CTE/Data/Master/Programs/HSPCTES2 /* CTEIMSDV.CTEIMSSTUMA /* /* DATA OUT: /data/projects/CTE/data/Master/Students/CTE_APPLICATIONS /* /* PRECEDING PROGRAM: /data/production/student/programs/master/STUCOHHSP20072018MA /* FOLLOWING PROGRAM: /data/projects/CTE/Master/CTE_REP1_SAMP /* /* REVISIONS /* PROGRAMMER DATE DESCRIPTION /* JJK 01/03/2019 MERGE CTE COURSE DATA /* JJK 02/20/2019 MATCH WITH UPDATED HSPCTES2 FILE /* THAT INCLUDES FLAGS FOR OTHER HSAP-CTE MATCHES /* JJK 09/13/2019 UPDATED WITH 2017 HSAP COHORT AND FOLLOW-UP THROUGH 2017-2018 /* JJK 02/13/2020 UPDATED WITH REVISED VERSION OF THE HSAP-CTE LINK FILE /* NOTE NEW VARIABLE NAMES ON HSAP-CTE LINK FILE /* NEW FILE NAME (CTE_APPLICATIONS) TO AVOID CONFUSION WITH PREVIOUS VERSION /* JS 06/26/2020 UPDATED CTESCHGRP TO SEPARATE DISCONTINUED HSAP-CTE FROM NEVER CTE /* WHS 06/26/2020 CHANGED INPUT DATSET TO UPDATED VERSION OF HSAP_APPS20072018 /* WHS 11/03/2020 CHANGED IF STATEMENT TO CTEPOSPERMAT=2 FOR MATCHENRDAN VARIABLE /* WHS 11/05/2020 CREATE MACRO VARIABLE FOR NSC VARIABLES AND ADDED IT TO DUMVR ARRAY /* WHS 11/11/2020 CREATE MACRO VARIABLE FOR CNY VARIABLES AND ADDED IT TO DUMVR ARRAY /* WHS 03/05/2021 MERGE APP DATA WITH IMS (I.E. INTERNSHIP) DATA /* WHS 04/08/2021 UPDATED DO STATEMENT IN %CTECRS MACRO TO INCLUDE 2019 CTE-COURSE DATA /* WHS 05/14/2021 ADD IF STATEMENT TO CTE COURSE CREDIT VARIABLES FOR INCOMPLETE COHORTS /* JJK 07/20/2021 UPDATED TO INCLUDE DATA THROUGH 2020 /* COMMENTED OUT SUPLEMENTAL CTE COURSE DATA PENDING FURTHER DISCUSSION /* WHS 08/04/2021 ADDED LABEL FOR MATCHPRGM: UNMATCHED AFTER MAIN ROUND /* /* COMMENTS: WE SHOULD MAKE IT CLEAR THAT MATCHPRG = MATCHPRGFIN WHEN NO MAIN ROUND MATCH /* ESTIMATED RUN TIME PER HSAP_APPS SORT: 6-8 MINUTES /* /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -*/ %PUT LAST RUN: &SYSDATE; OPTIONS NOCENTER PAGESIZE=62 LINESIZE=200 MERGENOBY=WARN ; OPTIONS ERRORS=50 ; OPTIONS NOMPRINT ; OPTIONS FMTSEARCH=(STUPROUT.STUFORMATS) ; OPTIONS NOSYMBOLGEN NOMLOGIC; **** Use to diagnose macro problems ; /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -*/ LIBNAME STUPROMA '/data/production/student/data/master'; LIBNAME STUPRORW '/data/production/student/data/raw'; LIBNAME STUPROS1 '/data/production/student/data/step1'; LIBNAME STUPROS2 '/data/production/student/data/step2'; LIBNAME STUPROUT '/data/production/student/util' ; LIBNAME SCHPROS1 '/data/production/school/data/step1'; LIBNAME CTEPRGDV '/data/projects/CTE/Data/development/CTEPRG'; LIBNAME CTESTUMA '/data/projects/CTE/Data/Master/Students'; LIBNAME CTESTUR1 '/data/projects/CTE/Data/Report 1'; LIBNAME CTEPRGMA '/data/projects/CTE/Data/Master/Programs'; LIBNAME CTECRSMA '/data/projects/CTE/Data/Master/Courses'; LIBNAME CTEIMSDV '/data/projects/CTE/Data/development/CTEIMS' ; LIBNAME DEVJJKCT '/data/development/jjk/CTE' ; TITLE1 'RESEARCH ALLIANCE FOR NEW YORK CITY SCHOOLS' ; TITLE2 'HSAP APPLICATION FILE WITH CTE PROGRAM INFORMATION' ; /*WHS NOTE: STUCRSCTES2 UPDATED TO 2019-2020 SCHOOL YEAR*/ %LET CTELASTYR = 2020 ; PROC FORMAT ; VALUE CTESCHTYPF 1 = '1:CTE HS' 2 = '2:ACADEMIC HS' ; VALUE CTEPRGCURF 0 = '0:PHASED OUT' 1 = '1:PERSISTED' 2 = '2:PHASED OUT' ; VALUE CTESCHGRPF 1 = '1:CTE HS (PERSISTING)' 2 = '2:ACAD HS (PERSISTING)' 3 = '3:PHASED OUT' 9 = '9:NO CTE' ; RUN; /* PREPARE HSAP APPLICATION DATA FOR MERGING WITH HSAP-CTE PROGRAM DATA */ %PUT SORT HSAP_APPS DATA BY APPLICATION PROGRAM; PROC SORT DATA=STUPROMA.HSAP_APPS20072018 OUT=CTE_APPS ; BY HSCOHG9A PROG ; RUN; ... View more

%PUT STATEMENT WONT PRINT TO LOG

by Quartz | Level 8 in SAS Programming
‎08-04-2021 12:03 PM
‎08-04-2021 12:03 PM
I have a program with a lot of huge sorting procedures.  I would like to print messages to the log so I can easily check on the progress of the program while it's running.  I have written something to similar to the code below but when I run it, nothing gets printed to the log even after the PROC SORT procedure is finished. %put SORT ED DATA BY KEYVAR; proc sort data = ED_DATA; by KEYVAR; run; ... View more

Re: proc mi produces no log

by Quartz | Level 8 in Statistical Procedures
‎07-20-2021 11:16 AM
‎07-20-2021 11:16 AM
Thanks, this was really helpful. I always appreciate your help on SAS communities as well as your blog posts. ... View more

Re: proc mi produces no log

by Quartz | Level 8 in Statistical Procedures
‎07-14-2021 01:20 PM
‎07-14-2021 01:20 PM
ETHCAT has 5 and the other three variables have 2, so there are a total of 40 groups I suppose (5x2X2X2=40)     The FREQ Procedure ETHCAT Frequency Percent Cumulative Frequency Cumulative Percent 2: ASIAN/PACIFIC ISLANDER 8496 16.99 8496 16.99 3: HISPANIC 19228 38.46 27724 55.45 4: BLACK, NON-HISPANIC 14359 28.72 42083 84.17 5: WHITE, NON-HISPANIC 7300 14.60 49383 98.77 8: OTHER RACE/ETH 617 1.23 50000 100.00 GENCAT Frequency Percent Cumulative Frequency Cumulative Percent 1: FEMALE 25793 51.59 25793 51.59 2: MALE 24207 48.41 50000 100.00 OVERAGEG09 Frequency Percent Cumulative Frequency Cumulative Percent 0 39900 79.80 39900 79.80 100 10100 20.20 50000 100.00 REGMTHP65G08P Frequency Percent Cumulative Frequency Cumulative Percent 0 39062 78.12 39062 78.12 100 10938 21.88 50000 100.00 ... View more

Re: proc mi produces no log

by Quartz | Level 8 in Statistical Procedures
‎07-14-2021 09:10 AM
‎07-14-2021 09:10 AM
PROC MI DATA = HSPSTUID(OBS = 50000) NIMPUTE = 1 OUT = HSPSTU_MI1; CLASS ETHCAT GENCAT OVERAGEG09 REGMTHP65G08P; FCS NBITER = 1 REGPMM ( ATTPCTROLG08MI); FCS NBITER = 1 REG (ELASSCZG08MI); FCS NBITER = 1 REG (MTHSSCZG07MI); FCS NBITER = 1 REGPMM ( ATTPCTROLG07MI); FCS NBITER = 1 REG (ELASSCZG07MI); FCS NBITER = 1 REG (MTHSSCZG08MI); VAR ETHCAT GENCAT OVERAGEG09 REGMTHP65G08P ATTPCTROLG08MI ELASSCZG08MI MTHSSCZG07MI ATTPCTROLG07MI ELASSCZG07MI MTHSSCZG08MI; WHERE IN_MI = 1; RUN ... View more

Re: proc mi produces no log

by Quartz | Level 8 in Statistical Procedures
‎07-13-2021 06:58 PM
‎07-13-2021 06:58 PM
Hi,    I'm having a similar issue with a dataset with 750,000 observations.  I'm only imputing 6 variables, and I'm using REG for four of those.  However, I wanted to use REGPMM for two attendance rate (percentage) variables because using REG results in numbers over 100.  I've tried using the MAX option before but that didn't work.    Is there anyway to increase the efficiency of this operation?  If not, is there a way to estimate the length of time that is needed?  I've tried gradually increasing the number of observations, but the effect on computation time doesn't seem linear.   Thanks, Bill ... View more

Re: PROC MI: Predicitive Mean Matching is Running slowly on Large Data...

by Quartz | Level 8 in Statistical Procedures
‎07-13-2021 05:04 PM
‎07-13-2021 05:04 PM
There are  757,400 observations.   The code below takes about a minute with 50,000 observations.  However, the procedure seems to take for ever (at least 2 hours) after going beyond about 70,000 observations, which is only 1/10 of the data.   PROC MI DATA = HSPSTUID(OBS = 50000) NIMPUTE = 1 OUT = HSPSTU_MI1; CLASS ETHCAT GENCAT OVERAGEG09 REGMTHP65G08P; FCS NBITER = 1 REGPMM ( ATTPCTROLG08MI); FCS NBITER = 1 REG (ELASSCZG08MI); FCS NBITER = 1 REG (MTHSSCZG07MI); FCS NBITER = 1 REGPMM ( ATTPCTROLG07MI); FCS NBITER = 1 REG (ELASSCZG07MI); FCS NBITER = 1 REG (MTHSSCZG08MI); VAR ETHCAT GENCAT OVERAGEG09 REGMTHP65G08P ATTPCTROLG08MI ELASSCZG08MI MTHSSCZG07MI ATTPCTROLG07MI ELASSCZG07MI MTHSSCZG08MI; WHERE IN_MI = 1; RUN None of the variables in the class statement have any missing data.   The two attendance variables use REGPMM and the other four continous variables use REG.  I've order the FCS statements by the number of missing observations in each variable.    I purposedly set NIMPUTE  = 1.  I know multiple imputation is superior but we're currently sticking with single imputation.  In any case, I imagine multiple imputation would take even longer.     Thanks for your help.   ... View more

PROC MI: Predicitive Mean Matching is Running slowly on Large Dataset

by Quartz | Level 8 in Statistical Procedures
‎07-13-2021 04:38 PM
‎07-13-2021 04:38 PM
I'm trying to impute 6 variables using PROC MI and the  fully condidional specification method.     Two of these variables are attendance  rates bounded by 0 and 100.   I have used PROC MI and FCS to imputed values which works fine except for the fact that some attendance values end up being a lot more than 100.  I've tried setting MAX =100 but the algorithm ends up stopping.  I then tried to use predictive mean matching for the attendance variables.  However, the dataset is very large so this seems to take a very long time.  I was wondering if there was any way to speed up predictive mean matching.  ... View more

Re: Calculating AUC on test/validation dataset

by Quartz | Level 8 in SAS Programming
‎06-29-2021 02:19 PM
‎06-29-2021 02:19 PM
Okay, re-read and now realized where I was getting confused.  Just had a hard time understanding that you could fit the model and calculate prediction scores for different datasets in the same PROC LOGISTIC step.    This is ultimately  the fastest way to compare training/test AUC.   proc logistic data=train; model y(event="1") = x1 x2; score data=train fitstat; score data=valid fitstat; run; .  ... View more

Re: Calculating AUC on test/validation dataset

by Quartz | Level 8 in SAS Programming
‎06-29-2021 02:09 PM
‎06-29-2021 02:09 PM
Thanks, I actually read that but was confused on two points. First it seems that having INMODEL as a separate step is unnecessary because I can just add a score statement to the first PROC LOGISTIC step. I just wanted to confirm that that is correct. proc logistic data = XTRAIN outmodel= MODEL1 ; model Y (EVENT = '1')= X1 X2 ; score data = XTEST out =YTEST (rename = (P_1 = YPRED)); run; Second I don't understand why you need the model statement in the second PROC LOGISTIC step since you already fit the model in the first step. proc logistic data= YTEST; model Y(event="1")=; roc pred =YPRED; ods select ROCOVERLAY; run; ... View more

Calculating AUC on test/validation dataset

by Quartz | Level 8 in SAS Programming
‎06-29-2021 01:06 PM
‎06-29-2021 01:06 PM
  I'm trying to calculate the AUC on a holdout test (or validation) data set.  My model has a pretty good AUC on the training data (.87), but I would like to see if it performs well out of sample.    Let's say original datset contains three variables Y, X1, and X2.  I split this dataset into two smaller datasets: XTRAIN and XTEST.       These are the steps I have done.   First I trained my model on the training dataset XTRAIN proc logistic data = XTRAIN outmodel= MODEL1 ; model Y (EVENT = '1')= X1 X2 ; run;   Next I use my model to make predictions on the test dataset.   proc logistic inmodel = MODEL1 ; score data = XTEST out = YPRED_test (rename = (P_1 = YPRED)); run; Next I use these predictions to plot ROC and calculate my test AUC   proc logistic data= YPRED; model Y(event="1")=; roc pred =YPRED; ods select ROCOVERLAY; run; I just wanted to check if these steps were correct.  In general, these are the steps for out-of-sample model validation I have used when programming in R and Python.    ... View more

Re: PROC TABULATE use variable names instead of labels

by Quartz | Level 8 in SAS Programming
‎04-09-2021 11:59 AM
‎04-09-2021 11:59 AM
Thanks, this worked!   I was thinking there would be a NOLABELS options for PROC TABULATE as there is for PROC MEANS but there evidently isn't.  I guess this actually makes sense because it would be rare that you would prefer the variable name when using PROC TABULATE.  In any case, your solution worked.            ... View more

PROC TABULATE use variable names instead of labels

by Quartz | Level 8 in SAS Programming
‎04-09-2021 10:56 AM
‎04-09-2021 10:56 AM
Is there way to use the variable names in PROC TABULATE instead of labels? ... View more

ODS PDF

by Quartz | Level 8 in SAS Programming
‎08-04-2020 12:24 PM
‎08-04-2020 12:24 PM
Hi,   I'm trying to output a pdf with plots from sgpanel with a title on the first page.  However, when I try to add a title, it ends up on the last page.  I would like the title to appear on the first page.  I know I can use ODS TEXT, but I would like the title to have multiple lines.   Here is my code.   GOPTIONS DEVICE=PNG; ods pdf file = '/FILEPATH/PLOT.pdf'; title 'My Plot'; title2 'My subtitle'; title3 'My second subtitle'; proc sgpanel data = HAVE; panelby group_variable; scatter x = x_variable y = y_variable; run; title; title2; title3; ods pdf close; Thanks,   Bill ... View more

Re: PROC SURVEYREG: Find Grand Mean at certain value of X when you hav...

by Quartz | Level 8 in Statistical Procedures
‎08-03-2020 09:06 PM
‎08-03-2020 09:06 PM
That is a good question.  There is some weighting involved, and we are also clustering standard errors.   In the previous example, I was trying to simplify the code a little bit, but the actual code looks more like this.     PROC SURVEYREG DATA = HAVE; CLUSTER DISTRICT; CLASS SCHOOL; MODEL GRADUATION = SCHOOL RANK / SOLUTION; WEIGHT WEIGHTS; RUN; ... View more
Contact Me
Online Status
Offline
Date Last Visited
‎11-20-2023 06:04 PM
Likes given to
Likes from
Quick links