I'm not a statistician, just a user of statistics, but:
1. Are the proportions very different with the smaller and larger samples?
2. Is this an inference issue rather than a statistics issue per se? I recently was reading Nakagawa and Cuthill (below); they make the point (and I've seen other versions of this illustration) that, if one could measure the lengths of wings of all the birds in a population, the difference in mean wing length between sexes would almost certainly be statistically significant. Whether this would mean something to biologists is another question.
Nakagawa, S., and I. C. Cuthill. 2007. Effect size, confidence interval and statistical significance: A practical guide for biologists. Biological Reviews 82: 591-605.
... View more