Interpretive Guide for the Achievement Levels Report (2003

Text Preview:
Interpretive Guide for the Achievement Levels Report
                                        (2003 Revision)

                           ITBS/ITED Testing Program


The purpose of this Interpretive Guide is to provide information to individuals who will use the
Achievement Levels Report for monitoring the achievement of student grade groups, both at the
building level and system-wide, and for reporting the progress of those groups to others. This
document describes what is on the report, how this reporting service was developed, and how the
report information can be used to accomplish each of several purposes.

                                   What Is This Report?

The Achievement Levels Report is a building/system report of achievement based on scores from
the Iowa Tests of Basic Skills and the Iowa Tests of Educational Development. It is provided as
a standard-service report for each of grades 4, 8, and 11 when there are at least 10 students
per grade level in the processing order. One report is for mathematics, based on the ITBS
Mathematics Total scores or the ITED Math Concepts and Problems scores. A second report,
for reading, is based on the ITBS and ITED Reading Comprehension scores. The report for
science in grades 8 and 11 only is based on the ITBS and ITED Science scores. This service is
not available for any other grades or for scores in any other test areas of the ITBS or ITED.
The report also is provided for subgroups of students on the basis of gender, racial/ethnic group,
free or reduced-price meal eligibility, migrant status, IEP status, and status as an English
Language Learner (ELL).

                            How Can This Report Be Used?

This reporting service was designed primarily (a) to help school districts monitor the
achievement of grade groups of students as part of their school improvement plans and (b) to
facilitate the reporting of achievement results of school buildings and the district to the local
community, the school board, and the Iowa Department of Education. Specifically, the report
might be used to:

1.   describe the achievement levels of 4th, 8th, and 11th grade students this year in terms of
     content skills, rather than only comparing their scores with the scores of their peers
     throughout the state. The focus of this kind of interpretation is on the content expertise of
     the studentswhat they are able to do in relation to the content expectations (or standards)
     established by your district, as measured by the tests.

2.   monitor year-to-year changes in the percentages of students scoring at the very lowest or
     very highest parts of the score scale rather than only comparing average scores from year to
     year. Assuming comparable groups from year to year, this type of report makes it possible
     to view the changes in achievement of the very lowest- or very highest-achieving subgroups
     in a grade instead of just looking at how the achievement of the average student in that
     grade may have changed.

3.   facilitate reporting achievement data to the Iowa Department of Education as required by
     Chapter 12 of the Iowa Code or by one or more federally funded programs. Some federal
     guidelines call for accountability datascores that speak to improvement and that relate to
     schools attaining adequate progress toward locally established achievement goals. This
     report is intended to help schools report such information when the ITBS or ITED has been
     chosen by the local district as one of the assessment tools for this purpose.
4.   report student achievement to your community and school board or to prepare news
     releases for your local newspaper (as may be appropriate to meet state or federal guidelines
     for local involvement in school improvement efforts).

It is also important to understand what this reporting system is not intended to be. For
example, it was not developed to help interpret the scores of individual students. The
achievement levels and descriptions of them are meant to be used to characterize the
performance of groups of students. Within any grade group, there are likely to be students
whose individual performance does not necessarily fit the description that has been attached to
their achievement-level group. Individual students should not be classified as Skilled or Weak
or Accomplished: these are labels used for convenience to describe subgroups of students who
have demonstrated a particular level of test performance.

Another significant aspect of this reporting system to note is that the report describes how
students actually performed, not how they should have performed or how they are expected to
perform. In other words, there are no performance standards built into this reporting scheme.
Any standards used to decide whether student performance is "good enough" should be
developed and imposed locally by the community.

                   How Was This Reporting System Developed?

The achievement levels that form the basis for this report were developed using an approach
called "achievement level benchmarking." This approach involves partitioning the score scale
from a norm-referenced test like the ITBS and ITED into achievement regions that, altogether,
cover the entire achievement continuum. Then, actual test-item responses of students in the
national standardization sample are used to describe the specific achievement of students who
scored within each region. Benchmarking of this type involves establishing a baseline of
performance and using it as a basis for comparing student performance in subsequent years.
Such comparisons show whether student groups in successive years are gaining, remaining
steady, or losing ground relative to the baseline performance. The baseline performance also
can be used along with long-term goals to help establish annual improvement expectations.
Then year-to-year changes can be interpreted in light of the expectations that had been
established for reaching the long-term goals.

For this particular reporting system, the baseline performance for the set of achievement levels
was the set of scores obtained by a national standardization sample of grades 4 and 8 ITBS
test takers and grade 11 ITED test takers. The samples are nationally representative groups
who were tested with Form A in the spring of 2000. (Previous reports based on scores from
Forms K/L were based on 1992 national norms.)

Achievement Levels Were Defined

The national percentile rank score scale was partitioned in two ways to create two sets of
achievement levels. For the first set, achievement levels were defined by the percentile-rank
groupings 1-40, 41-89, and 90-99. The use of these three regions meets the reporting
requirements of Title I, as described by current federal guidelines and Iowa code (Chapter 12):
achievement should be reported as Less-than-Proficient, Proficient, and Advanced as defined by
state ("local" in the case of Iowa) performance standards. For the Achievement Levels Report,
these three regions were labeled by Iowa Testing Programs as Low Performance, Intermediate
Performance, and High Performance.

The set of three achievement levels described above may be useful for meeting state/federal
reporting responsibilities of local districts, but they are probably too broad for monitoring local
achievement, for determining just where changes are or are not being made within the full
achievement range. Consequently, an expanded set of regions was defined to allow for greater

                                                                                                      2
precision in viewing the changes in achievement of student subgroups within a district.
Essentially, each of the three regions was subdivided into a pair of regions to form a six-level
system. The national percentile rank groupings formed in this way were:

           Levels       NPR Ranges                           Levels         NPR Ranges
           Weak            1-9                              Skilled           76-89
          Marginal        10-40                          Accomplished         90-94
          Moderate        41-75                          Distinguished        95-99

For both of these sets of achievement-level labels, names used by Title I and those used by the
National Assessment of Educational Progress (NAEP) were avoided intentionally. Because this
achievement level benchmarking approach does not build performance standards into it (what
students should be able to do), as do those used by some states for Title I or those used by
NAEP, unique names for Iowa achievement levels were important for reducing possible
inappropriate comparisons across reporting systems.

Achievement Level Descriptions Were Created

The same general procedures were used in mathematics, reading, and science to create
performance descriptions for each achievement level in the grades included. For the sake of
brevity, only the mathematics procedures for grades 4 and 8 will be detailed here. Within each
of the nine groupings of students noted above (the three original ones and the six formed by
subdividing them), the proportion (p) of students correctly answering each question was
computed. Then within each grouping, these p-values were clustered according to the content
skill measured by the item. (For example, all geometry items were grouped together, all single-
step problem solving items were grouped together, and so on.) The median p-value was
calculated for each skill cluster, and then a judgment was made about each skill according to
the following guide:
      Median p          Code                    Descriptors
       .00-.39          N       Can't do the skill, rarely can do, or seldom can do
       .40-.69          B       Beginning to develop; sometimes can do; does some
       .70-.89          D       Is developing skill; usually or often can; does most
       .90-.99          Y       Can do it (implies always, all, and every)

The letter codes formed the basis for writing content descriptions for each of the nine
achievement levels. For this purpose, mathematics skills were grouped so that statements
could be limited to performance with regard to four broad skill areas: concepts, estimation,
problem solving, and data interpretation. Upon completion of these descriptions, a second
person used the descriptions to translate back to the four letter symbols. Differences in the
classifications resulting from this reverse translation task were cause for making small
adjustments in the wording of the original content descriptions. (All descriptors are listed in
the Appendix.)

The Stability of the Results Was Checked

Because the scores used for this work were from spring testing with Form A, but the results
will be used statewide at all times of year (for the ITBS) and with both Forms A and B, it was
necessary to ensure that the relationships found in the content descriptions also were accurate
for Form B and for fall and midyear testing. Therefore, the procedures noted above for creating
descriptions were followed with the Form A 2000 fall standardization sample data. The
results showed that descriptions for spring and fall for the same test form were consistent in
pattern and that the fall p-values were lower, as should be expected. The same procedure was
carried out with the Form B fall standardization sample data. Some minor adjustments to the
original descriptions were made as a result of the Form B replication.


                                                                                                   3
               How Can the Achievement Levels Report Be Used?

Locating Information on the Report

The sample report for Mathematics, which is attached to this Guide, is used here to identify
the various segments of the report and to describe how those parts relate to one another. In
the upper right corner, information about system name and Iowa code, test date, test form
used, and processing order number are given. This area is separated from the rest of the report
by two banner lines that show the two sets of achievement level labels and how these regions
differ from one another in size. For example, it can be seen that the Intermediate Performance
level is in the middle, and it occupies the largest portion of the full score range. Further, the
Moderate and Skilled levels are subdivisions of the Intermediate Performance level, and the
Moderate level is both the lower and the larger of the two.

The bar graph in the lower left shows how students in the fourth grade of this system were
distributed throughout the entire score range. In particular, 21.9% are in the Low Performance
level, most of those being in the Marginal portion. About 60% exhibited Intermediate
Performance, most in the Moderate level, and 17.1% were in the High Performance region, with
about two-thirds of those being in the Distinguished level. The exact percentages are listed in
the columns of numbers that separate the bar graph from the descriptions of achievement.

The right part of the page provides written descriptions of the types of math skills the students
in each of the six achievement levels are typically able to demonstrate, as measured by the
ITBS Math Total score. These descriptions are constant for all Iowa schools for each grade
level and subject area combination: the description for Moderate, for example, is the same for
the fourth-grade groups in math for all systems in the state. The written descriptions for the
set of three broader achievement levels are the same for all Iowa fourth-grade groups in math,
regardless of the school system.

For the subgroup report based on students with IEPs, the written descriptions are not printed.
The descriptors for each grade were developed with the test level that is most often given for
the grade: Level 10 for grade 4, Level 14 for grade 8, and Level 17/18 for grade 11. Because
many of the students with IEPs are tested out of level, usually with a lower test level than is
used in their grade, the descriptions for the on-level-test would not be appropriate for describing
their performance.

Describing Achievement Status

The achievement status of a given grade group can be seen most readily by examining the bar
graph. The graph shows such things as: where the largest concentration of students has
scored; how large the Low Performance group is and whether most of the students in it are
near the top or near the bottom of that region; and how large the High Performance group is
and where the largest concentration of scores is located within that region.

The percentages on the bars, and those in the middle columns of the report, describe how the
grade group distributed itself along the score continuum. There is no established set of
numbers with which these should be compared, and there is no set of numbers for the
"average" Iowa school system with which the numbers can be or should be compared. This
report encourages the use of criterion-referenced interpretations rather than norm-referenced
ones. (See the ITBS or ITED interpretive guide for further explanation of these two types of
score interpretation.) That is, the report helps describe the kinds of knowledge and skill a
group of students have demonstrated through their test performance within an area like
reading or mathematics. The report users must decide whether this performance is sufficient,
acceptable, extraordinary, or on target for meeting the school system's preset performance
goals.
                                                                                                  4
For the most part, schools will benefit from using the set of six achievement levels with the
school staff in trying to understand the levels of student achievement. The set of three levels is
more effective for inclusion in an Annual Progress Report (APR) or for reporting to the
community or school board. The set of three levels is a more gross level of reporting, and the
percentages associated with it are likely to be more stable than those from the six-level set. Of
course, the percentages reported with both sets of achievement levels contain a certain amount
of error, in one sense because the scores of the individuals within each grouping contain some
measurement error. However, usually the error is not likely to be large enough to distort the
general impression about the distribution of achievement among students in a building or
system grade group.

In differentiating the performance of groups in any of the six achievement levels, it is important
to realize that differences between adjacent groups are qualitatively quite small. For example,
the differences between students in the Moderate and Skilled groups are far smaller than those
between students in the Moderate and Accomplished groups. Most certainly some students
who are part of the Skilled level would end up in the Moderate group if the students were to be
retested, and vice versa. This is one good reason for not using the achievement levels for
classifying individual students.

Monitoring Changes Each Year

Because the Achievement Levels Report is prepared for only grades 4, 8, and 11, reliance on this
report alone to check district or building achievement status and monitor change is inadvisable.
Schools also should be concerned with the progress students make prior to the end of fourth
grade, and four years is probably too long to wait to determine whether new instructional
strategies implemented in each of grades 4-8 are having their intended effects. Annual testing
in all grades provides the information necessary to monitor change in a timely manner so that
indications of program ineffectiveness can be obtained before too many students have
progressed too far through the program. Scoring services that have been available to Iowa
schools for a number of years can assist in monitoring change on an annual basis with every
grade. For example, the Frequency Distribution service provides building and system reports
that show how students in various regions of the score scale have performed. In addition, the
Group Narrative Summary report consists of bar graphs for building and system grade groups
in each test area. The Interpretive Guide for School Administrators has sample copies of
reports from both of these services and further information about them can be obtained by
calling the Iowa Testing Programs staff.

When the Achievement Levels Report is available for a particular grade level for two or more
consecutive years, change can be examined by comparing the distribution of percentages from
each successive year. Several related questions are reasonable to pose in the context of
monitoring change in achievement:

       How much have scores changed from last year?

       Were there differences in the amount of change among the high achievers, low achievers,
       or middle achievers?

       Were the changes of a size that would have been expected based on the achievement
       goals that had been established?

       Were the changes in the direction predicted, given the general achievement level of the
       current group relative to that of last year's group?




                                                                                                 5
Download Link:
Share Link: Forum Link:

More on Business

  • Picture: Gr 10 Maths Literacy - Paper 1

    Gr 10 Maths Literacy – Paper 1

    File Size: 249.96 KB, Pages: 10, Views: 348,184 views

    GRADE 10 EXEMPLAR EXAMINATION NOVEMBER 2006 MATHEMATICS LITERACY PAPER 1 Minimum time: 1 hours Maximum time: 2 hours 75 marks PLEASE READ THE FOLLOWING CAREFULLY 1. This paper consists of: 5 questions an answer sheet with grid paper for question 4 (d). 2. Answer all …
  • Picture: Improving cash flow using credit management - CIMA

    Improving cash flow using credit management – CIMA

    File Size: 374.27 KB, Pages: 28, Views: 332,095 views

    Improving cash flow using credit management The outline case sponsored by Improving cash flow using credit management sponsored by Albany Software focuses on developing award-winning software to transform financial processes and is the market leader in electronic payment solutions. Albany makes financial transfers and subsequent …
  • Picture: MATERIAL SAFETY DATA SHEET - compressorparts.com

    MATERIAL SAFETY DATA SHEET – compressorparts.com

    File Size: 311.77 KB, Pages: 8, Views: 152,565 views

    Product Name: MOBIL RARUS SHC 1025 Revision Date: 16Feb2007 Page 1 of 8 _____________________________________________________________________________________________________________________ _ MATERIAL SAFETY DATA SHEET SECTION 1 PRODUCT AND COMPANY IDENTIFICATION PRODUCT Product Name: MOBIL RARUS SHC 1025 Product Description: Synthetic Base Stocks and Additives Product Code: 606343-00, 973764 Intended Use: …
  • Picture: Onan RV Generator Troubleshooting Guide - Flight Systems, Inc.

    Onan RV Generator Troubleshooting Guide – Flight Systems, Inc.

    File Size: 2,729.64 KB, Pages: 18, Views: 29,064 views

    RV GENERATOR TROUBLESHOOTING GUIDE A Service of FLIGHT SYSTEMS 207 Hempt Road Mechanicsburg, PA 17050 Ph: 717 590 7330 Fax: 717 590 7327 www.flightsystems.com CONTENTS RV Generator Troubleshooting Overview 1 RV Generator FAQs 5 Onan RV Generator Charts: Applications Chart 8 Test Information for Troubleshooting …
  • Picture: Hot Spot Hot Tubs and Spas Owners Manual-2010

    Hot Spot Hot Tubs and Spas Owners Manual-2010

    File Size: 2,385.04 KB, Pages: 44, Views: 22,814 views

    Watkins Manufacturing Corporation congratulates you on your decision to enjoy the finest spa available... Welcome to the growing family of Hot Spot Spa owners. Owner's Manual This Owner's Manual will acquaint you with your new spa's operation and general maintenance. We suggest that you take …

Leave a Reply

Your email address will not be published. Required fields are marked *