Although the items of the Positive and Negative Syndrome Scale (PANSS) are ordinal, continuous data methods are consistently used to analyze them. The current study addresses this issue by applying a categorical method and critically examining the ideas of item inclusion and goodness of fit. Data from 1527 subjects were used to test a proposed solution to the factor structure of the PANSS using a categorical factor analytic method. The model was made more generalizable by setting a minimum level of association between the item and the factor, and the results were then compared to existing solutions. The model was also tested for consistency in a first-episode sample. Use of categorical methods indicated similar results to previous analyses; however, it is demonstrated that the strength of the estimates can be unstable when items are shared across factors. The current study demonstrates that solutions can change substantially when a model is over-fitted, and therefore use of measures of fit as the criterion for an acceptable model can mask important relationships and decrease clinical validity.