NH-pace

New Hampshire 2018-2019 PACE Pilot Program: Disaggregated Data Visualization

Can Performance-Based Assessments Adequately Display Student Achievement Gaps?

_{Author: Taylor M. True, March 2024}

New Hampshire - PACE Program (Background)

In spring of 2015, New Hampshire piloted its innovative “Performative Assessment of Competency Education” (PACE) initiative, a project-based alternative to standardized assessments. Beginning in 2014, New Hampshire was granted a waiver from the U.S. Department of Education, relieving the state from certain assessment accountability requirements under the Every Student Succeeds Act (ESSA). New Hampshire’s PACE pilot was subsequently granted continued support from the USDOE through the 2018-19 school year per participation in the newly established federal Innovative Assessment Demonstration Authority (IADA) provisions. The pilot was conducted from 2014 through 2020, servicing an initial nine districts participating with full-implementation in SY 2015-2016 and thirteen districts participating in SY 2019-2020. The following discussion and analysis of PACE will consider two periods of implementation: SY14-16 and SY18-19.

PACE was designed for grades K-12 and delivered in grades 3-12, and employs a standards-based, project-based, practical and on-demand approach to measuring student skill attainment in core subject areas. The initiative’s implementation and performance measures contain two components: local performance assessment tasks created by individual teachers and implemented throughout the year, and common tasks formulated as state-wide tasks to offer a competency comparison between districts. This project-based approach to summative assessments poses an alternative to federally-compliant standardized testing systems, such as New Hampshire’s Smarter Balanced Assessment Consortium, 2015-17, and the New Hampshire Statewide Assessment System (SAS), 2018-current. To ensure reliability and comparability of PACE assessment scoring and measurement, participating school districts were to continue administering New Hampshire’s statewide summative assessments once in an elementary school grade, once in a middle school grade, and one grade in high school.

Group-level disaggregated scores were published and available to reveal student PACE performance by various socio-demographic indicators, e.g. race, economically disadvantaged, and IEP status. The current PACE 2018-2019 Performance Evaluation Report does not include formal analysis of student performance by group-level indicators. To appropriately visualize and display student 2018-2019 PACE results for the proportion of students scoring “Proficient or Above” in Math (mat) and English Language Arts (rea), computer vision technology was used to convert published data (portable document format) to Excel, then manually transformed using R Studio:

As of 2022, the New Hampshire Department of Education announced its premature suspension the PACE pilot under IADA, due to stated concerns with PACE assessment task administration, data collection and management, and the resulting validity of PACE assessment results. Thus, I caution against the use of these results in informing subsequent equity-based decisions or the future of PACE or similar initiatives. The purpose of this visualization is provided as a supplementary visualization tool based on publicly-available PACE scoring data. Its intended use is such that, should assessment item and item-scores yield acceptable validity, reliability, and generalizability measurement indices, such performance-based scoring can indeed be visualized and utilized to determine achievement gaps between student socio-demographic groups.

_{(Note: this brief is excerpted from a report prepared for Democrats for Education Reform (DFER) - Massachusetts Chapter. This excerpt is prepared by Taylor M. True from the Harvard Graduate School of Education, 2023. The contents of this brief are not intended for public use, nor do they claim to represent DFER, the Harvard Graduate School of Education, or any associated individuals or institutions aside from the primary author of this excerpt. Full report available upon request.)}