Publications

This paper presents and illustrates a framework for visualizing large-scale assessment results in a dynamic score reporting interface to support teachers in making content-referenced interpretations of student growth. The reporting interface maps student performance to locations along a research-based learning progression to facilitate interpretations of the quantitative differences along a vertical score scale. We illustrate content-referenced growth reporting in the context of a learning progression for how students understand fractions. Two key aspects of the illustration include evidence showing that discrete levels of the learning progression have a moderate to strong association with the difficulty of assessment items that were coded to the levels, and results from a small-scale pilot test of the reporting interface with practicing classroom teachers. We speculate about important aspects of implementation that would strengthen the validity of CRG score reporting interpretations and uses.

Spatial reasoning comprises a set of skills used to mentally visualize, orient, and transform objects or spaces. These skills, which develop in humans through interaction with our physical world and direct instruction, are strongly associated with mathematics achievement but are often neglected in early grades mathematics teaching. To conceptualize ways to increase the representation of spatial reasoning skills in the classroom, we examined the outcomes of cognitive interviews with kindergarten through grade two students in which they engaged with one spatial reasoning task. Qualitative analyses of students’ work samples and verbal reasoning responses on a single shape de/composition task revealed evidence of a continuum of sophistication in their responses that supports a previously articulated hypothetical learning progression. Results suggest that teachers may be able to efficiently infer students’ skills in spatial reasoning using a single task and use the results to make instructional decisions that would support students’ mathematical development. The practical implications of this work indicate that additional classroom-based research could support the adoption of such practices that could help teachers efficiently teach spatial reasoning skills through mathematics instruction.

Research-practice partnerships are an increasingly popular approach for supporting the use of research to inform and guide efforts to improve and transform education. To date, however, evaluators have lacked measures to evaluate such partnerships, which typically aim for a range of outcomes. This paper describes a project to develop validity evidence for a survey to evaluate the effectiveness of research-practice partnerships in education. The project followed an evidence-centered design approach to developing and evaluating the validity of the survey measure, collecting and analyzing data from 65 different research-practice partnership. Results indicate the scales were reliable overall, but that educators and researchers exhibited different response patterns to items. Evidence from this study does not support summative uses of the scales for evaluating individual RPPs; however, they could be used formatively for informing discussions about improvement. Future research can investigate the possibility that the scales could be used in evaluating portfolios of RPP projects.

An increasingly popular form of collaboration involves forming partnerships among researchers, educators, and community members to improve or transform education systems through research inquiry. However, not all partnerships are successful. The field needs valid, reliable, and useful measures to help with assessing progress towards partnership goals. In this community case study, we present a participatory, mixed-methods approach designed to create measures for assessing progress of research-practice partnerships (RPPs) in the field of education. The case illustrates a novel approach to participatory measurement design, driven by needs from the field. As a result, the measures align with the values and practices of the very collaborations the measures were intended to assess.

This study uses structural equation modeling to investigate the relationships between pre-service teachers’ (PSTs’) course-taking history, beliefs about mathematics, beliefs about students’ mathematical ability, and opinions about (1) how student errors should be addressed when they occur and (2) how much emphasis should be given to various forms of assessment. The results indicate that some types of courses are strongly associated with beliefs about the nature of mathematics. Specifically, PSTs who have taken more mathematics courses are more likely to see mathematics as a set of rules. PSTs’ views about mathematics strongly predict their beliefs about who can do high-level mathematics, how they think student errors should be addressed, and which forms of assessment they think are most appropriate. Implications for teacher preparation programs are discussed.

The early development of spatial reasoning skills has been linked to future success in mathematics (Wai, Lubinski, & Benbow, 2009), but research to date has mainly focused on the development of these skills within classroom settings rather than at home. The home environment is often the first place students are exposed to, and develop, early mathematics skills, including spatial reasoning (Blevins-Knabe, 2016; Hart, Ganley, & Purpura, 2016). The purpose of the current study is to develop a survey instrument to better understand Kindergarten through Grade 2 students’ opportunities to learn spatial reasoning skills at home. Using an argument-based approach to validation (Kane, 2013), we collected multiple sources of validity evidence, including expert review of item wording and content and pilot data from 201 parent respondents. This manuscript outlines the interpretation/use argument that guides our validation study and presents evidence collected to evaluate the scoring inferences for using the survey to measure students’ opportunities to learn spatial reasoning skills at home.

Classroom assessment research in the United States has shifted away from the examination of teacher-made tests, but such tests are still widely used and have an enormous impact on students’ educational experiences. Given the major shifts in educational policy in the United States, including the widespread adoption of the Common Core State Standards, I argue that researchers should examine the tests and quizzes that teachers create and administer in order to determine whether those policies have had the intended impact on teachers’ assessment practices. Furthermore, these inves­tigations should be grounded in discipline-specific conventions for developing and demonstrating knowledge. I then propose a research-based framework for analysing mathematics exams that focuses on alignment with learning goals, cognitive complex­ity, variety of task formats, attentiveness to culture and language, and clarity of expectation. This framework is meant to be used formatively, helping researchers, administrators, and teachers iden­tify strengths and areas for growth.

High-school mathematics teachers tend to use computational, constructed response questions in their classroom tests. However, the rapid shift to distance learning resulting from the COVID-19 pandemic created technological obstacles to using these items.  This study investigated teachers’ reasons for using particular items and how they adapted their assessment practices during distance learning.  Teachers reported using computational items because they provided insight into students’ thinking, helped with determining point allocations, and prevented academic dishonesty.  During distance learning, most sample teachers maintained their use of these items by collecting students’ written work via uploaded photographs or a “whiteboard” feature in a paid assessment system. Despite continued use of computational questions, few teachers reported using shown work to determine student understanding or to assign partial credit.  Instead, they relied heavily on auto-grading and reported collecting student work mostly to deter cheating.  Implications of these findings and future directions for research are discussed.

There is renewed interest around including performance assessments in state and local assessment systems to spur positive changes in classroom instruction and student learning. Previous research has identified the external conditions that mediate the role of assessment in changing instructional practices. We extend that work by focusing on the internal classroom conditions that support improvements in student learning. We identified six key instructional practices from three teacher quality frameworks that may result from policy changes that include complex, performance-based assessments. For each practice, we explored the bidirectional relationships among the instructional core of students, teachers, and content. We argue that altering these relationships requires teachers and students to have both the disposition and the capacity to change, and we identify the assumptions that need to hold in order for those changes to occur in response to the inclusion of performance assessments in state and/or local assessment systems.

Related CenterLine Blog Post (Part 1)

Related CenterLine Blog Post (Part 2)