West Virginia Assessment

Attention: open in a new window.Print

WVDE Plan For Assessment Alignment of Standards For Implementing Frameworks for 21st Century Learning

Introduction:

With the focus on School Improvement and 21st Century Skills for West Virginia schools, the Department of Education is releasing a Request for Proposal in November 2005 to transform the West Virginia Measures of Academic Progress, our statewide assessment program, into 21st century assessments that measure and that are aligned to learning skills that encompass problem solving, critical thinking, information and communications skills, directional skills and ICT Literacy which will be embedded in the West Virginia Content Standards and Objectives by grade level. Research shows that by using assessment data aligned to 21st century skills to inform educators of instructional areas of growth and change, they are better prepared to determine and meet student needs for the 21st century.

Objectives:

As we work towards school improvement and closing the achievement gap, statewide consistency of purpose and systemic reform are major goals. Meeting these goals will help to improve student learning for all subgroups as we work to close the achievement gap. The objectives of this new assessment program are to create valid and reliable assessments 1) that are aligned to the 21st century skill descriptors and state content standards and objectives, 2) that inform instruction, 3) that promote school improvement and 4) that produce results that can be used to calculate school, county and state accountability as per the No Child Left Behind Act of 2001.

Tactics:

The following specifies the early 2006 development of each of the following assessments:

  1. Name of assessment/grade level of administration
  2. Type of assessment
  3. Description/purpose of assessment
  4. Uses and decision-making data for each assessment
  5. Development timeline

Online Formative Assessment Item Bank (Grades 3-12) are criterion-referenced assessments that provide the following:

  1. Electronically delivered student testlets designed to assist students outside of the school environment to promote thinking and learning. These would also be accessible by parents and teachers. The assessments are aligned to the West Virginia Content Standards and Objectives (CSO’s) and will have the 21st century skills embedded by grade level.
  2. Online teacher item bank and testlets available for teacher use as formative classroom assessments aligned to 21st Century West Virginia Content Standards and Objectives (CSO’s) that he/she taught during that instructional period or cycle.
  3. Online district level benchmark assessment items and testlets available for school district benchmark assessments aligned to the 21st Century West Virginia Content Standards and Objectives (CSO’s). Results from these assessments will redirect or refocus district benchmark instruction.

K-2 Assessments (K-2) is a criterion-referenced test that measures student mastery of the 21st Century West Virginia Content Standards and Objectives (CSO’s) by the content area and grade level. The purpose of K-2 assessments is to measure student mastery of the CSO's and thinking skills including problem solving, creative thinking, and information and communications skills. The assessment results are used to re-direct instruction and assist in school improvement efforts in reading/language arts, mathematics, science and social studies. New development is scheduled to begin in June 2006.

WESTEST (Grades 3-8 and 10) is a criterion-referenced test that measures student mastery of the 21st Century West Virginia Content Standards and Objectives (CSO’s) by the content area and grade level. The purpose of WESTEST is to measure student mastery of the CSO's and thinking skills including problem solving, creative thinking, and information and communications skills. The assessment results are used to re-direct instruction and assist in school improvement efforts in reading/language arts, mathematics, science and social studies. Additionally, WESTEST results are used to calculate accountability to meet the NCLB accountability requirements in grades 3-8 and 10 in reading/language arts and mathematics. CTB/McGraw-Hill will continue as the vendor for WESTEST in grades 3-8. New development is scheduled to begin in June 2006.

ACT EXPLORE is a norm-referenced assessment that measures skill levels for English, mathematics, reading, and science as well as problem solving, information and communications skills, and directional skills for 8th grade students. In addition, information about students' educational career plans, interest, high school course work plans and self-identified needs for assistance are gathered and reported. West Virginia State Board of Education Policy 2510: Assuring the Quality of Education Regulations for Education Programs requires that career awareness, exploration activities and ACT EXPLORE results be used by 8th graders to develop their individualized plans for the 9th and 10th grades. The purpose of this assessment is to provide career awareness and exploration activities; additionally, EXPLORE is used by 8th graders to develop their individualized plans for the 9th and 10th grades. Assessment results assist students/parents/educators in decision-making about educational career plans, interest and high school course work plans needed for college admission. ACT will continue as the vendor for ACT EXPLORE at grade 8.

Online Writing Assessment (Grades 3 - 11) are criterion-referenced tests of the West Virginia Writing Standard and the related objectives, which are part of the West Virginia CSO’s by grade level. The purpose of this assessment is to promote the improvement of communication and writing skills of students in all grades by assessing narrative, descriptive, persuasive and informative prompts. The Writing Assessment results are used to make instructional decisions and assist in school improvement efforts in reading and writing. New development is scheduled to begin in June 2006.

WESTEST/College Admissions Assessments (Grades 9–11) will be a combination of a college admissions test and a criterion-referenced test which will measure skill levels of English, mathematics, reading, science, social studies for 9th, 10th, and 11th grade students for college admissions. All assessments will include a writing assessment component designed to enhance the student’s ability to apply problem solving, creative thinking, and information and communication skills. The tests will be augmented and aligned to the 21st Century West Virginia Content Standards and Objectives (CSO’s). The assessment will be used as the measure of instructional need in grades 9,10 and 11 and as a predictive tool to determine college performance. Grade 11 results will be used as the NCLB measure for school, county and state accountability. New development is scheduled to begin in June 2006.

National Assessment Educational Progress (Grades 4, 8) is a criterion-referenced test of the national NAEP Standards. NAEP assesses specific academic areas in students who are randomly selected in grades 4, 8 and 12. NCLB requires NAEP reading and mathematics assessments to be administered to fourth and eighth grade students every two years. These assessments are aligned to the NAEP standards and performance levels. There are no individual student, school or county results provided for this assessment. The purpose of this assessment is to assess and report what students know and should be able to do. NAEP assessments results will be used as an informal validation of the state assessment results in grades 4 and 8 for reading, mathematics and science and program evaluation. The federal government pays for development and administration of this program; ETS has the current federal contract.

Metrics:

The Office of Instruction plan includes an internal gap analysis study of the current CSO’s to the new 21st Century West Virginia CSO’s for grades 1 - 12 for reading, mathematics, science, and social studies. The Office of Assessment will conduct an internal alignment study involving West Virginia teachers in the selection and/or development of test items that align to the new 21st Century West Virginia CSO’s. Additionally, the assessment office will contract for an external alignment study to determine alignment of the accountability operational tests to the curriculum standards in reading/language arts, mathematics, science and social studies for all grades assessed by the accountability tests. This alignment study is designed to produce measures on five criteria. The underlying assumption of this approach is to compare the relationship between assessment instruments and standards by analyzing how these documents compare using the same criteria. The five criteria are as follows:

  1. Categorical Concurrence
    This criterion between standards and assessment is met if the same or consistent categories of content appear in both documents.
  2. Depth-of-Knowledge Consistency
    This criterion between standards and assessment is met if what is elicited from students on the assessment is as demanding cognitively as what students are expected to know and do as stated in the standards.
  3. Range-of-Knowledge Correspondence
    This criterion is met if a comparable span of knowledge expected of students by a standard is the same as, or corresponds to, the span of knowledge that students need in order to correctly answer the assessment items/activities.
  4. Balance of Representation
    This criterion is met if the degree to which one objective is given emphasis on the assessment is comparable to the emphasis given.
  5. Source of Challenge
    This criterion is met if the primary difficulty of the assessment items is highly related to students’ knowledge and skill in the content area as represented in the standards.

An Alignment Study Report will be generated that summarizes the findings of the evaluators by content area. The report will determine if the assessments are acceptably aligned using the above five criteria.

Secondly, the Standard Setting process will be a modified version of the Bookmark Standard Setting Procedure (BSSP). The Standard Setting will be held in Charleston, West Virginia and will use West Virginia grade level educators/participants in the process.

Educators/participants will be recruited from across the state of West Virginia to recommend the cut scores. The West Virginia Department of Education (WVDE) will recruit approximately 24 participants in each grade/content area. WVDE will then divide the participants within each grade/content area into four groups that are balanced in terms of relevant demographic characteristics, such as geographic location and school size.

Participants at the Standard Setting will recommend performance-level cut scores for Graded 3, 6, 8, and 10 Reading/Language Arts, Mathematics, Science and Social Studies. In each grade/content area, panelists will be engaged in three or four rounds of activities in which they will recommend four cut scores that defined five performance levels: Novice, Partial Mastery, Mastery, Above Mastery, and Distinguished. Next, selected participants from each content area will convene to recommend well-articulated cut scores such that similar percentages of students would be classified in each performance level across grades within any given content area.

A Standard Setting Report will be generated that summarizes the cut scores recommended by participants in each grade/content area after the final round of discussion and voting to include a summary of the cross-grade committee smoothed data. This report will provide the impact data that are associated with the cut scores presented in the document. Impact data will be the performance of the all students and subgroups from the pilot testing.

Partnership for 21st Century Skills