After 18 months of pilot implementation, we have begun full implementation of the NWEA MAP assessment. The MAP assessment is a computer-based adaptive assessment which describes what students are ready to learn next in reading, writing, and mathematics. The MAP assessment is administered three times a year for all students in grades K-8. See the BRSU Local Assessment Plan for a description of the NWEA MAP as well as the other assessments administered to students.
Why Implement the NWEA MAP?
1) Transition to Personalized Learning
Our district, like many districts around the world, is attempting to reconcile the new ability to personalize learning for students with an industrial model of schooling. We felt it was necessary to have a valid external measure of student learning in place while we made this transition to a personalized learning system. We believe NWEA MAP supports personalized learning by measuring student growth on an equal increment scale irrespective of assigned student grade levels. It also provides data to students and teachers in a format which readily supports student goal setting, a key component for personalizing learning for students.
2) Support Continuous Organizational Improvement
We assess student learning to help answer the fundamental question from our Ends policies, “how are students doing,” and to then decide next steps in the learning process. Efficient and effective assessment data are able to function at three organizational levels: the instructional level, the administrative level, and the policy level. At the instructional level, assessment data provide teachers with formative data. Formative data give teachers immediate feedback on student learning in order for them to modify instructional approaches and to design programs for remediation or acceleration. At the administrative level, assessment data help evaluate instructional systems such as professional development and curriculum development. At the policy level, assessment data provide assurance to board members that district Ends for student learning are being met. The NWEA MAP assessment was selected because it is a single assessment which can satisfy the assessment data needs of the organization at all of these levels.
3) Increase the Validity, Reliability, and Efficiency of Assessment
Time is our most precious instructional resource. When instruction is interrupted for the purpose of assessment, we need to ensure the interruption is worth it. Unfortunately, we have developed several assessments in Vermont which are very time consuming but which do not necessarily provide us with results that can justify the loss in instructional time.
One such assessment is the PNOA, or Primary Number and Operation Assessment, which can take a classroom teacher up to 45 minutes to administer one-on-one to each student. Although the PNOA can provide a teacher with useful formative data, it takes considerable time away from regular instruction to administer. The PNOA is an example of an assessment which is “expensive” in terms of instructional time, relatively complex to administer (each teacher needs training to administer the assessment in order to obtain valid results), and relatively unreliable in terms of obtaining consistent data across a system due to its relatively low level of inter-rater reliability. The PNOA is representative of several assessment approaches in Vermont education which have become “best practice” due to their sophistication rather than their validity or reliability.
In the enclosed article by Tim Shanahan, the author discusses this phenomena where the most sophisticated tool can become to be considered the best tool in a profession. He makes the comparison between the medical profession and education. Following this analogy, the PNOA might considered to be a CAT scan and the NWEA MAP a chest x-ray because the PNOA is delivered in an artisanal manner, one student at a time, whereas the NWEA MAP is
administered to many students at the same time using computers. In fact, if one were to
compare the instructional-level data produced by these two assessments, it is clear the NWEA MAP is the CAT scan and the PNOA is the x-ray, and a relatively unreliable one at that since the PNOA can not compare to the superior validity and reliability of the NWEA MAP.
It is our expectation that some of our Vermont formative assessments such as the PNOA will be abandoned in favor of using NWEA MAP and thereby free up instructional time. These
decisions will need to be made through an understanding of the value of the assessment data relative to the impact on instructional time.
Another key aspect of NWEA MAP efficiency is the facile manner in which it allows for the
collection and organization of assessment data. Since it is a computer-based assessment,
MAP assessment results can obtained immediately after the test, and can be organized based on any number of building-level testing groups. Most of our Vermont local assessments require the manual collection and arrangement of data which further contributes to their delay in producing actionable results.
4) The Political Context of Assessment
The NECAP will be replaced by the SBAC assessment in the near future. This will create a gap in our external assessment data for several years. Like NWEA MAP, the SBAC assessment will be a computer-based adaptive assessment so it should produce results in a more timely
manner. The SBAC has not been finalized, however, and will take a few years to stabilize. This uncertain political context for assessment led us to conclude we needed an external
assessment to bridge the gap between NECAP and SBAC. NWEA MAP is a well established
assessment which should serve us well in this purpose.