ANOVA not giving p value

[jkwong]

I'm trying to do an ANOVA for the question: does the search term used (4 different categorical) affect readability (continuous)? There are 100 rows total, with each term having 25 values. Readability is measured in 8 different ways (8 variables) but I want to assess all four search terms versus one readability variable at a time.

I input the dataset but when I used this command: 

proc glm data= mydata;
class searchterm;
model words = searchterm ;
means searchterm/hovtest welch;
run;

there was no result/ANOVA tables. It just showed my dataset in the outcome tab. 

I tried creating a new dataset of the average of the 25 hits for each search term (so there were only 4 rows, one for each search term), but the ANOVA test that resulted didn't have any p values. (Maybe because you need to have n>30 or assume normal distribution?) I also have a feeling this isn't the right way to use an ANOVA but I don't know how to do it using the first dataset, which I think is the correct way to input the data

Screenshots for the dataset made of averages: 

Even trying to run a Kruskal-Wallis test for both datasets yielded the same results (no tables for the first version and no p values for the second)

Command used (adjusted for each dataset):

proc npar1way data=mydata wilcoxon dscf;
class searchterm;
var words;
run;

Thanks in advance!

[sbxkoenk]

Hello,

Typically, zero degrees-of-freedom (df=0) for Source=Error means you only have one observation per  unique combination of class levels.

The degrees of freedom (DF) column should be used to check the analysis results. The model degrees of freedom for a one-way analysis of variance are the number of levels minus 1; in your case, 4 – 1 = 3. The Corrected Total degrees of freedom are always the total number of observations minus one; in your case 4 (??) – 1 = 3. The sum of Model and Error degrees of freedom equals the Corrected Total.

I can only conclude there's a problem with your dataset (only 4 observations??).

When the degree of freedom is zero, there is no way to affirm or reject the model! In this sense, the data have no "freedom" to vary and you don't have any "freedom" to conduct research with this data set.

SAS/STAT 15.3 User's Guide
The ANOVA Procedure
One-Way Layout with Means Comparisons
https://go.documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/statug/statug_anova_gettingstarted01.htm

BR,

Koen

[PaigeMiller]

I tried creating a new dataset of the average of the 25 hits for each search term (so there were only 4 rows, one for each search term)

Once you average the values for each search term, and wind up with four rows, then the model fits perfectly, 0 df for error, no statistical tests possible. I think you want to run the ANOVA on the un-averaged data, 25 hits for each search term.

(Maybe because you need to have n>30 or assume normal distribution?)

Not the case at all.

[jkwong]

@PaigeMiller the first method I described was using the un averaged data with 25 hits per search term, and that didn't yield any ANOVA tables when I ran the code. It just came up with the data I had input

[PaigeMiller]

Hello, @jkwong 

You can't just say the words "didn't yield any ANOVA table" because we need to see (a portion of) the data and we need to see the ENTIRE log from PROC GLM. Without that, there's no way anyone can help you.

Please provide the data as working SAS data step code (examples and instructions) and not in any other form. Please copy the log as text and paste it into the window that appears when you click on the </> icon here.

[jkwong]

Hello @PaigeMiller I tried to follow along with the link you provided, but I'm not sure if I did so correctly (hence why I need all this help in the first place haha). I attached the log as code (the </> vox), and the data set I used/my proc glm code. Thanks again and thanks for your patience!

 1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 68         
 69         /*try this by creating my own dataset lol*/
 70         
 71         data mydata;
 72         input searchterm $ website $ words $ sentences $ wps $ GFI $ CLI $ FKGL $ ARI $ SMOG $ FRE;
 73         datalines;
 
 NOTE: SAS went to a new line when INPUT statement reached past the end of a line.
 NOTE: The data set WORK.MYDATA has 100 observations and 11 variables.
 NOTE: DATA statement used (Total process time):
       real time           0.00 seconds
       user cpu time       0.00 seconds
       system cpu time     0.00 seconds
       memory              677.28k
       OS Memory           19876.00k
       Timestamp           10/29/2023 01:40:22 PM
       Step Count                        61  Switch Count  2
       Page Faults                       0
       Page Reclaims                     135
       Page Swaps                        0
       Voluntary Context Switches        10
       Involuntary Context Switches      0
       Block Input Operations            0
       Block Output Operations           264
       
 
 175        ;
 176        
 177        run;
 178        
 179        proc glm data= mydata;
 180        class searchterm;
 181        model words = searchterm;
 ERROR: Variable words in list does not match type prescribed for this list.
 NOTE: The previous statement has been deleted.
 182        run;
 
 183        
 184        /*the issue above is that I finally formatted the data correctly into tables, but I can't seem to get an output for the
 184      ! anova test when I try to run the command*/
 185        
 186        
 187        /*maybe the issue is that I'm using the wrong test lmao like I'm prob supposed to be using the Kruskal Wallis test bc
 187      ! n<30 for each group?*/
 188        
 189        /*proc npar1way data=mydata wilcoxon dscf;
 190         class searchterm;
 191          var words;
 192        run;*/
 193         /*above value is the kruskal wallis but that also just goes to the output data still*/
 194        
 195        
 196        /*going to try to do this stuff based on what the SAS forum person told me but we shall see if it works
 197        uploaded the data2datastep code so going to try to see if it works here*/
 198        
 199        
 200        
 201        
 202        
 203        OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 213        

/*try this by creating my own dataset lol*/

data mydata;
input searchterm $ website $ words $ sentences $ wps $ GFI $ CLI $ FKGL $ ARI $ SMOG $ FRE;
datalines;

Circumcision https://medlineplus.gov/circumcision.html 203 15 13.53 14.67 12.65 10.73 9.86 13.1 42.24
Circumcision https://www.hopkinsmedicine.org/health/treatment-tests-and-therapies/circumcision 946 60 15.77 12.52 10.04 10.29 8.64 12.17 49.35
Circumcision https://www.urologyhealth.org/urology-a-z/c/circumcision 675 45 15 11.69 9.8 9.63 8.41 11.45 52.74
Circumcision https://www.acog.org/womens-health/faqs/newborn-male-circumcision 1138 81 14.05 11.81 10.34 9.57 8.21 11.68 51.48
Circumcision https://kidshealth.org/en/parents/circumcision.html 906 51 17.76 12.4 10.11 10.39 9.52 11.94 52.91
Circumcision https://caringforkids.cps.ca/handouts/pregnancy-and-babies/circumcision 821 55 14.93 11.91 9.97 9.65 8.25 11.49 52.47
Circumcision https://www.nyp.org/healthlibrary/special/circumcision 2508 199 12.6 10.61 9.28 8.95 6.83 10.77 53.35
Circumcision https://www.mayoclinic.org/tests-procedures/circumcision/about/pac-20393550 873 53 16.47 13.55 12.11 11.33 10.58 13.07 43.2
Circumcision https://en.wikipedia.org/wiki/Circumcision 10518 444 23.69 18.02 13.61 15.1 14.94 16.31 28.98
Circumcision https://www.betterhealth.vic.gov.au/health/conditionsandtreatments/circumcision 1311 54 24.28 16.88 12.81 14.48 14.58 15.22 34.48
Circumcision https://www.ncbi.nlm.nih.gov/books/NBK535436/ 1250 82 15.24 15.54 14.82 12.97 12.25 14.49 29.2
Circumcision https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2801794/ 1989 73 27.25 18.8 13.4 16.19 16.42 16.96 27.55
Circumcision https://www.webmd.com/sexual-conditions/circumcision 895 47 19.04 13.78 10.5 11.16 10.38 13.14 48.93
Circumcision https://www.childrensdayton.org/patients-visitors/services/urology/conditions/circumcision 1002 52 19.27 12.54 8.95 10.48 9.24 11.73 54.21
Circumcision https://www.ucsfhealth.org/education/the-circumcision-decision 675 44 15.34 12.06 9.55 9.8 8.07 11.97 52.14
Circumcision https://www.healthychildren.org/English/ages-stages/prenatal/decisions-to-make/Pages/Circumcision.aspx 781 55 14.2 11.21 10.01 9.2 8 11.19 54.42
Circumcision https://www.uptodate.com/contents/circumcision-in-baby-boys-beyond-the-basics 2123 118 17.99 14.85 13.52 12.82 12.35 14.17 35.19
Circumcision https://raisingchildren.net.au/guides/a-z-health-reference/circumcision 977 55 17.76 14.39 12.25 11.99 11.23 13.85 40.73
Circumcision https://www.healthline.com/health/circumcision 1973 107 18.44 13.84 11.35 11.91 10.8 13.44 42.5
Circumcision https://www.childrenshospital.org/treatments/circumcision 1702 99 17.19 14.28 11.52 11.57 10.41 13.42 42.7
Circumcision https://healthcare.utah.edu/mens-health/treatments/circumcision-frenulotomy 1115 67 16.64 13.8 11.75 11.25 10.36 13.06 44.04
Circumcision https://www.manchestercircumcisionclinic.com/procedures-4 267 14 19.07 12.72 10.82 11.51 10.65 12.71 46.48
Circumcision https://med.stanford.edu/newborns/professional-education/circumcision/plastibell-technique.html 121 8 15.12 14.31 12.88 11.37 10.66 13.61 40.46
Circumcision https://my.clevelandclinic.org/health/treatments/16194-circumcision 1317 124 10.62 11.48 10.41 8.93 7.09 11.02 49.98
Circumcision https://familydoctor.org/circumcision/ 862 59 14.61 10.99 8.43 8.89 6.9 11.13 57.35
Kid https://www.seattlechildrens.org/conditions/a-z/circumcision-problems/ 953 74 12.88 7.75 7.3 6.31 5.35 8.48 72.77
Kid https://www.cedars-sinai.org/blog/is-circumcision-good-or-bad.html 602 37 16.27 13.95 11.59 10.81 10.08 13.23 46.56
Kid https://www.stanfordchildrens.org/en/topic/default?id=circumcision-90-P03080 1281 96 13.34 10.08 8.15 7.92 6.19 10.2 62.07
Kid https://www.chp.edu/our-services/urology/patient-procedures/circumcision 1568 85 18.45 12.12 9.04 10.19 8.96 11.57 54.84
Kid https://childrensnational.org/visit/conditions-and-treatments/urology/circumcision 1120 78 14.36 11.35 10.19 9.6 8.21 11.55 51.84
Kid https://www.acog.org/womens-health/faqs/newborn-male-circumcision 1138 81 14.05 11.81 10.34 9.57 8.21 11.68 51.48
Kid https://www.healthychildren.org/English/ages-stages/prenatal/decisions-to-make/Pages/Circumcision.aspx 640 56 11.43 9.38 7.42 7.21 4.95 9.67 63.71
Kid https://www.childrensdayton.org/patients-visitors/services/urology/conditions/circumcision 1002 52 19.27 12.54 8.95 10.48 9.24 11.73 54.21
Kid https://kidshealth.org/en/parents/circumcision.html 906 51 17.76 12.4 10.11 10.39 9.52 11.94 52.19
Kid https://utswmed.org/medblog/baby-boy-circumcision/ 1080 59 18.31 14.03 10.27 11.16 9.88 13.13 47.65
Kid https://www.urmc.rochester.edu/encyclopedia/content.aspx?contenttypeid=90&contentid=p03080 1661 129 12.88 9.75 7.89 7.65 5.82 9.97 63.12
Kid https://caringforkids.cps.ca/handouts/pregnancy-and-babies/circumcision 821 55 14.93 11.91 9.97 9.65 8.25 11.49 52.47
Kid https://www.mayoclinic.org/tests-procedures/circumcision/about/pac-20393550 873 53 16.47 13.55 12.11 11.33 10.58 13.07 43.2
Kid https://www.betterhealth.vic.gov.au/health/conditionsandtreatments/circumcision 1311 54 24.28 16.88 12.81 14.48 14.58 15.22 34.48
Kid https://www.ucsfhealth.org/education/the-circumcision-decision 675 44 15.34 12.06 9.55 9.8 8.07 11.97 52.14
Kid https://raisingchildren.net.au/guides/a-z-health-reference/circumcision 977 55 17.76 14.39 12.25 11.99 11.23 13.85 40.73
Kid https://www.childrenshospital.org/treatments/circumcision 1702 99 17.19 14.28 11.52 11.57 10.41 13.42 42.7
Kid https://med.stanford.edu/newborns/professional-education/circumcision/plastibell-technique.html 121 8 15.12 14.31 12.88 11.37 10.66 13.61 40.46
Kid https://www.webmd.com/parenting/baby/circumcision-what-expect 765 54 14.17 9.8 7.74 8.01 6.17 9.95 62.85
Kid https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4441785/ 2276 153 14.88 13.65 11.55 11.4 9.49 13.44 39.82
Kid https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8654051/ 4305 214 20.12 17.79 15.36 14.81 14.74 16.2 24.68
Kid https://my.clevelandclinic.org/health/treatments/16194-circumcision 1317 124 10.62 11.48 10.41 8.93 7.09 11.02 49.98
Kid https://www.nhs.uk/conditions/circumcision-in-boys/ 889 50 17.78 12.56 9.89 10.56 9.35 12.2 50.99
Kid https://myhealth.alberta.ca/Health/aftercareinformation/pages/conditions.aspx?hwid=tw12604 640 56 11.43 9.38 7.42 7.21 4.95 9.67 63.71
Kid https://www.hopkinsmedicine.org/health/treatment-tests-and-therapies/circumcision 946 60 15.77 12.52 10.04 10.29 8.64 12.17 49.35
Pediatric https://www.cedars-sinai.org/blog/is-circumcision-good-or-bad.html 602 37 16.27 13.95 11.59 10.81 10.08 13.23 46.56
Pediatric https://www.stanfordchildrens.org/en/topic/default?id=circumcision-90-P03080 1281 96 13.34 10.08 8.15 7.92 6.19 10.2 62.07
Pediatric https://childrensnational.org/visit/conditions-and-treatments/urology/circumcision 1120 78 14.36 11.35 10.19 9.6 8.21 11.55 51.84
Pediatric https://www.chp.edu/our-services/urology/patient-procedures/circumcision 1568 85 18.45 12.12 9.04 10.19 8.96 11.57 54.84
Pediatric https://www.acog.org/womens-health/faqs/newborn-male-circumcision 1138 81 14.05 11.81 10.34 9.57 8.21 11.68 51.48
Pediatric https://www.healthychildren.org/English/ages-stages/prenatal/decisions-to-make/Pages/Circumcision.aspx 781 55 14.2 11.21 10.01 9.2 8 11.19 54.42
Pediatric https://www.childrensdayton.org/patients-visitors/services/urology/conditions/circumcision 1002 52 19.27 12.54 8.95 10.48 9.24 11.73 54.21
Pediatric https://www.pennstatehealth.org/childrens/services-treatments/circumcision 248 17 14.59 14.38 14.13 11.8 11.45 13.38 36.47
Pediatric https://kidshealth.org/en/parents/circumcision.html 906 51 17.76 12.4 10.11 10.39 9.52 11.94 52.19
Pediatric https://caringforkids.cps.ca/handouts/pregnancy-and-babies/circumcision 821 55 14.93 11.91 9.97 9.65 8.25 11.49 52.47
Pediatric https://www.uptodate.com/contents/circumcision-in-baby-boys-beyond-the-basics 2123 118 17.99 14.85 13.52 12.82 12.35 14.17 35.19
Pediatric https://www.ncbi.nlm.nih.gov/books/NBK535436/ 1250 82 15.24 15.54 14.82 12.97 12.25 14.49 29.2
Pediatric https://publications.aap.org/pediatrics/article/130/3/e756/30225/Male-Circumcision 16449 732 22.47 17.67 14.55 15.1 15.14 16.42 26.81
Pediatric https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6179567/ 4305 214 20.12 17.79 15.36 14.81 14.74 16.2 24.68
Pediatric https://www.aafp.org/pubs/afp/issues/2020/0601/p680.html 2009 101 19.89 15.42 12.4 12.64 12.27 14.71 39.89
Pediatric https://memorialhermann.org/services/treatments/pediatric-circumcision 388 21 18.48 15.64 11.94 12.84 11.29 14.28 35.89
Pediatric https://www.cincinnatichildrens.org/health/c/circumcision 1133 74 15.31 11.63 9.53 9.61 8.05 11.64 53.46
Pediatric https://www.luriechildrens.org/en/specialties-conditions/newborn-circumcision/ 633 40 15.82 13.34 11.91 11.85 10.16 12.99 38.28
Pediatric https://www.childrensdmc.org/services/urology/newborn-circumcision-clinic 495 25 19.8 15.76 12.63 13.35 12.42 14.7 34.63
Pediatric https://www.hopkinsallchildrens.org/Services/Pediatric-General-Surgery/Procedures/Circumcision 548 40 13.7 13 10.93 10.25 8.55 12.53 45.96
Pediatric https://ucipediatricurology.com/specialties/newborn-circumcision/ 642 31 20.71 17.63 13.94 14.47 13.87 15.6 28.21
Pediatric https://www.childrenshospital.org/treatments/circumcision 1702 99 17.19 14.28 11.52 11.57 10.41 13.42 42.7
Pediatric https://med.stanford.edu/newborns/professional-education/circumcision/plastibell-technique.html 121 8 15.12 14.31 12.88 11.37 10.66 13.61 40.46
Pediatric https://www.webmd.com/parenting/baby/circumcision-what-expect 765 54 14.17 9.8 7.74 8.01 6.17 9.95 62.85
Pediatric https://www.hopkinsmedicine.org/health/treatment-tests-and-therapies/circumcision 946 60 15.77 12.52 10.04 10.29 8.64 12.17 49.35
Child https://www.seattlechildrens.org/conditions/a-z/circumcision-problems/ 953 74 12.88 7.75 7.3 6.31 5.35 8.48 72.77
Child https://www.cedars-sinai.org/blog/is-circumcision-good-or-bad.html 602 37 16.27 13.95 11.59 10.81 10.08 13.23 46.56
Child https://www.stanfordchildrens.org/en/topic/default?id=circumcision-90-P03080 1278 96 13.31 10.05 8.1 7.88 6.14 10.18 62.25
Child https://www.chp.edu/our-services/urology/patient-procedures/circumcision 1568 85 18.45 12.12 9.04 10.19 8.96 11.57 54.84
Child https://www.acog.org/womens-health/faqs/newborn-male-circumcision 1138 81 14.05 11.81 10.34 9.57 8.21 11.68 51.48
Child https://www.healthychildren.org/English/ages-stages/prenatal/decisions-to-make/Pages/Circumcision.aspx 781 55 14.2 11.21 10.01 9.2 8 11.19 54.42
Child https://www.childrensdayton.org/patients-visitors/services/urology/conditions/circumcision 1002 52 19.27 12.54 8.95 10.48 9.24 11.73 54.21
Child https://kidshealth.org/en/parents/circumcision.html 906 51 17.76 12.4 10.11 10.39 9.52 11.94 52.91
Child https://www.urmc.rochester.edu/encyclopedia/content.aspx?contenttypeid=90&contentid=p03080 1661 129 12.88 9.75 7.89 7.65 5.82 9.97 63.12
Child https://utswmed.org/medblog/baby-boy-circumcision/ 1080 59 18.31 14.03 10.27 11.16 9.88 13.13 47.65
Child https://caringforkids.cps.ca/handouts/pregnancy-and-babies/circumcision 821 55 14.93 11.91 9.97 9.65 8.25 11.49 52.47
Child https://www.mayoclinic.org/tests-procedures/circumcision/about/pac-20393550 873 53 16.47 13.55 12.11 11.33 10.58 13.07 43.2
Child https://www.betterhealth.vic.gov.au/health/conditionsandtreatments/circumcision 1224 47 26.04 17.28 12.77 15.01 15.36 15.43 33.8
Child https://www.uptodate.com/contents/circumcision-in-baby-boys-beyond-the-basics 2123 118 17.99 14.85 13.52 12.82 12.35 14.17 35.19
Child https://www.ucsfhealth.org/education/the-circumcision-decision 675 44 15.34 12.06 9.55 9.8 8.07 11.97 52.14
Child https://raisingchildren.net.au/guides/a-z-health-reference/circumcision 977 55 17.76 14.39 12.25 11.99 11.23 13.85 40.73
Child https://www.childrenshospital.org/treatments/circumcision 1702 99 17.19 14.28 11.52 11.57 10.41 13.42 42.7
Child https://med.stanford.edu/newborns/professional-education/circumcision/plastibell-technique.html 121 8 15.12 14.31 12.88 11.37 10.66 13.61 40.46
Child https://www.webmd.com/parenting/baby/circumcision-what-expect 765 54 14.17 9.8 7.74 8.01 6.17 9.95 62.85
Child https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4441785/ 2276 153 14.88 13.65 11.55 11.4 9.49 13.44 39.82
Child https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2128676/ 943 42 22.45 17.42 13.63 14.66 14.4 15.79 29.92
Child https://my.clevelandclinic.org/health/treatments/16194-circumcision 1317 124 10.62 11.48 10.41 8.93 7.09 11.02 49.98
Child https://www.nhs.uk/conditions/circumcision-in-boys/ 889 50 17.78 12.56 9.89 10.56 9.35 12.2 50.99
Child https://myhealth.alberta.ca/Health/aftercareinformation/pages/conditions.aspx?hwid=tw12604 640 56 11.43 9.38 7.42 7.21 4.95 9.67 63.71
Child https://www.hopkinsmedicine.org/health/treatment-tests-and-therapies/circumcision 946 60 15.77 12.52 10.04 10.29 8.64 12.17 49.35
;

run;

proc glm data= mydata;
class searchterm;
model words = searchterm;
	run; 

[PaigeMiller]

 179        proc glm data= mydata;
 180        class searchterm;
 181        model words = searchterm;
 ERROR: Variable words in list does not match type prescribed for this list.
 NOTE: The previous statement has been deleted.

SAS is telling you that your variable WORDS is not numeric, and so you cannot use it in this code to produce ANOVA. The problem is caused by this:

data mydata;
input searchterm $ website $ words $ sentences $ wps $ GFI $ CLI $ FKGL $ ARI $ SMOG $ FRE;
datalines;

The dollar sign after WORDS indicates that you have character variables, you can't use character variables in your PROC GLM to the left of the equal sign in the MODEL statement, they must be numeric. Yet, somehow, in your original code, avgwords was numeric, so there is a disconnect here. How did you get numeric avgwords from character WORDS?

[jkwong]

@PaigeMiller I see my error, that was very helpful! Only searchterm and website were categorical variables, the rest were numerical. So I got rid of the "$" after the other variables and got p values for the ANOVA when I ran it again. Thank you so much!

[ballardw]

This message in your log

 NOTE: SAS went to a new line when INPUT statement reached past the end of a line.

often indicates the data set as created does not actually match your expectations. It means that you attempted to read more variables from an input line than exist and the first (or more) values from the following line are read into the last variables of the input statement.

So it is likely that some of your "searchterm" values are now actually Fre, Smog, Ari or something else. And then values after the actual searchterm are in other variables as well.

Also with everything using input $ the limit is 8 characters so Searchterm and Website are truncated to 8 characters.

Do not include blank lines in DATALINES blocks.