NAEP provides cross-state comparisons of student performance. Cross-state comparisons may yield different results, however, depending on the statistic that is used. The graphs below, which compare Idaho with her adjacent states on the NAEP 2007 Grade 8 Writing assessment, illustrate this not-so-uncommon phenomenon. The first comparison is based on overall student performance or NAEP average scale score.  The second is based on student performance relative to a cut score or NAEP percent at or above Basic (i.e., percent meeting or exceeding grade-level expectations).
 

Different results were observed when different statistics were used to describe and compare student performance on the NAEP 2007 Grade 8 Writing Assessment:

• When the Average Scale Score was used to compare Idaho with her adjacent states, one state (WY) scored higher than Idaho, three states (WA, MT and UT) were not different from Idaho, and one state (NV) was lower than Idaho.
   

• When the Percent At or Above Basic was used to compare Idaho with here adjacent states, no state scored higher than Idaho, three states (WA, MT and WY) were not different from Idaho, and two states (NV and UT) were lower than Idaho.

That different statistics can generate different results from one set of student data does not reflect on the technical quality of the test. The average scale score and percent above cut store statistics can be computed using data from any test, be the data from a high quality test or from a low quality test.

 The NAEP scale scores for all students in the sample are included in the calculation of the Average Scale Score, which is a measure of overall performance. This is not true of achievement level scores (i.e., Basic, Proficient, and Advanced). Only students whose NAEP scale scores are equal to or higher than the stipulated cut score for Basic count for Percent At or Above Basic. The performance of the very best students contributes no more to the percentage score than does the performance of students who are right at the cut score. On the other end of the spectrum, the lowest performing students have a negative impact on the percentage that is no different from the students who are only one point short of the cut score. 

The No Child Left Behind Act (NCLB) has focused the nation's attention on cut score analyses for the various state tests and for NAEP. NAEP does as-good-as or better job than any assessment in the world at setting cut scores for student performance standards. Nonetheless, NCLB requires that NAEP's achievement levels (i.e., Basic, Proficient and Advanced) be used only on a trial basis until the Commissioner of Education Statistics determines that the achievement levels are reasonable, valid, and informative to the public. The National Assessment Governing Board urges all who are concerned about student performance levels to recognize that the use of these achievement levels is a developing process and is subject to various interpretations. The Board and the National Center for Education Statistics believe that the achievement levels are useful for reporting trends in the educational achievement in the United States. Achievement level scores are being used on a trial basis and should be interpreted and used with caution.