In Part 4, we looked at an overview of scripted program research. Today, we need to look how those programs affect students. A critical aspect of any program is whether it produces positive results for all students in every demographic. The one-size-fits-all approach of scripted programs is supposed to be effective for all students. But there’s a large body of evidence that shows it isn’t best for everyone and may have some serious unintended effects.
One Size Does Not Fit All
The way Title I funding was restricted after 2002 meant that the schools most likely to implement scripted programs were those with high concentrations of students of color and students from economically disadvantaged families – typically, urban and rural (i.e. not suburban) schools. Teachers who work in low-income schools have been and still are most likely to face mandated use of scripted curricula1 and be required to adhere to it “with fidelity,” which means they aren’t allowed to adapt it even if it’s not meeting the needs of their students. This inability to adapt the program is highly problematic because there’s evidence that some of these programs don’t work for marginalized groups and actively inhibit the very approaches that are considered best practice for those groups.
- A study in California with 2nd through 5th grade students from 153 schools found that the scripted Open Court reading program did not promote higher early reading achievement among children from low socio-economic groups. Instead, their results showed that impoverished students in non-scripted programs outperformed students in schools using Open Court.
- One study found aspects of one program that might be considered discriminatory against poor students, special education students, and English Learners. The scripted literacy program Read Well focused heavily on decoding and minimally on comprehension, made moving students to higher groups very difficult, and relied heavily on contrived decodable literature that depressed engagement in reading across grade levels. Engagement is critically important for all students but particularly so for students who are struggling, such as special education students and language learners. One particular aspect of language learners that is relevant here is that they often learn to decode very well but don’t understand what they are reading. A program that doesn’t emphasize comprehension, then, does not address a critical need for these (or any) children.
- Another early study showed that while scripted programs may have produced gains in economically disadvantaged schools, they were only half as large as the gains produced by non-scripted programs. A 3-year study of Success for All (SFA) reading in Title I schools found that non-SFA schools averaged 17% gains in reading proficiency but those in SFA gained an average of just 8.5%. Although old (2001), this research is important because it is through Title I funding that NCLB required the adoption of “scientifically based” reading programs in 2002, so the students being underserved by SFA in the study were mostly from marginalized populations and this study showed that the “scientifically-based” scripted program produced only half the gains experienced in other schools.
- A phenomenological study of elementary students detailed multiple issues with an unnamed commercial scripted basal reading program that was designed to be “teacher-proof.”* Teachers had no control over grouping — reading level was determined by a single test and many students were placed below their actual reading level. Several teachers cited 5th grade students who had tested above the 70th and even 85th percentiles on the MAP test, but were placed in a 3rd grade-level intervention by the “scientifically-based”** program test. This test may be discriminatory for SPED students: one student with a lisp was identified for intervention because her incorrect pronunciations were counted as errors. Several teachers noted that the program diagnostic test did not allow for student self-correction – when a child reads something incorrectly and then fixes the error immediately. The program test counted these as errors, which affected the reading level the child was assigned to. Teachers were not allowed to override the results of the test.
- That same phenomenological study found that ELL students struggled with what the researchers called “forced phonics.” These students were confused by the contrived phonics texts that attempted to utilize as many words in the same word family as possible – jog, bog, dog, frog, log, cog, nog, etc. Such texts are often nonsensical to an English learner, but the program did not use any authentic texts. It could be argued that these texts are often just nonsensical for all children, full stop.***1
- Texas is in the midst of a state-wide push to adopt “high quality instructional materials” (a term that may or may not be accurate). The materials in question are largely scripted and tied to funding in such a way that districts are highly incentivized to use them rather than any of the many other resources Texas has specified over the years. Consider that Texas students are 53% Hispanic, 25% White, and about 13% Black. Additionally, 62% are economically disadvantaged, 24% are English language learners, 14% are special education, and 9% are gifted. The likelihood that such a limited set of resources can meet the needs of the entire body of extremely diverse students in Texas is so improbable as to be nonexistent.
Multiple studies have shown that implementation of scripted programs in diverse and economically disadvantaged schools (where they are most likely to be adopted) prevents or impedes culturally relevant pedagogy (CRP) – a methodology that recognizes and honors students’ cultures and experiences and is beneficial across all socioeconomic and cultural groups. Other things like state tests may also be impediments to CRP, but a scripted program specifically restrains the ability of the teacher to adapt instruction to include CRP. This is a loss for all students, but particularly for students of color.- A 2022 study from the Journal of Curriculum Studies found that the K-8 scripted ELA curriculum Wit and Wisdom centered Whiteness at every level of the curriculum: text selection, thematic grouping of texts, and essential questions for learning modules. The “overwhelming Whiteness” was so egregious that the researchers went beyond simply reporting it to offering ways to resist its central focus. A finding like this is important because all children – all – need to see themselves represented in the things they are asked to read and do, and they need exposure to ethnicities, backgrounds, and cultures that are not their own. This is often called “mirrors and windows” and it’s been shown to improve engagement and social-emotional understanding for all students, not just those of color. Engagement leads to better retention, which in turn improves achievement.
Depressing Critical Thinking and Problem Solving for Students
Many – probably most – schools assert that critical thinking and problem solving are important skills they want students to develop as part of their learning experience. Scripted curricula, however, may or may not offer practice with such skills. If the program is lacking higher order thinking and problem solving and the teacher is required to strictly adhere to the script, then the students simply don’t get those skills. The district’s vision and mission no longer matter; the program determines what students learn or don’t learn and how they practice that learning. There is some evidence that just being asked to use a scripted program can shift the focus of lessons away from critical thinking and deep conceptual understanding and toward more rote learning.
- Findings from a study of a scripted preschool math program showed that the teacher began using conversational patterns that resembled the scripted curriculum (i.e., prompts) when teaching math, and the focus shifted to correct answers versus reasoning or problem solving.
- A 2021 study evaluated teaching for creativity – defined as teaching students to be creative in their thinking – in Scottish elementary students. The study revealed that rigid teaching practices reduced the conditions for students that promoted creativity: autonomy, dialogue, collaboration, and flexibility. The authors noted that a “tick-box” mentality where the teacher was more concerned with making sure they covered the skills specifically inhibited teaching for creativity. The curriculum teachers were using was not scripted; rather, the researchers were making a connection between rigid teaching practices and the suppression of teaching for creativity. This finding is important because rigid teaching practices are a feature of scripted programs that are implemented with “fidelity.”
- The phenomenological study of elementary students recorded multiple teachers’ concerns that the program did not promote higher level thinking, something that was (and still is) critically important under the rigorous Common Core English language arts standards. Some teachers noted that this low-level thinking began to impact other content areas without scripted curricula, such as math.
- That same phenomenological study noted that after the scripted program was implemented, students seemed unable to take agency – they had trouble starting work, working with others in small groups, or progressing from task to task without specific direction. The scripted program seemed to produce dependent learners — students who need a teacher to direct every step of the learning. What should happen is that the teacher scaffolds learning up until the scaffold is no longer necessary and the student becomes independent. What seems to be happening with scripted programs is what Zaretta Hammond describes as “over-scaffolding and permanent instructional crutches. Students remain dependent learners; they never internalize cognitive routines and procedures. In contrast, an independent learner knows the mental operations that she needs to use.”2
- In the course of audits, we have observed multiple programs – scripted and not – that either don’t include best practice approaches or have misinterpreted a best-practice approach. One example is the way Eureka Math (EngageNY), which is heavily scripted, interprets the use of writing in math. Writing in math is highly desirable because it accesses different areas of the brain and improves retention, but the writing must be analytical or evaluative – analyzing another person’s errors, justifying how you know an answer is correct, recommending a course of action based on mathematical calculations, etc. But the primary form of writing students practice in Eureka Math is simply stating an answer in words rather than just numbers. So instead of looking at how a fictional student worked a problem and writing “Jan’s answer is incorrect. I know this because she did not line up the products the right way when she multiplied 146 and 254. The columns were wrong so she added them wrong. The right answer is 37,084,” they solve a problem and write “Max needed 482 balloons for the parade.” In terms of cognitive demand, these are not the same thing.
Higher order thinking skills – Analyzing, Evaluating, and Creating – are important for deep learning. And developing these skills requires a student-centered learning environment – one that is responsive to individual needs and progress. There’s quite a bit of evidence that scripted programs are not providing opportunities to develop those skills.
___________________________
*Again, I object to this insulting descriptor.
**Education throws around a lot of terms like this, assuming we all interpret them the same way. In fact, we often have very different ideas about what those terms mean. When we see poor tools like this that may be actively harming students, it’s clear we aren’t all on the same page.
***Consider this excerpt: “Cal had a cat. Mac was the cat. Mac had a bad mat. The mat had a sag. Mac had a bad nap. Mac was a tad sad. Cal had a pad. The pad had a bad tag.” Words like mat, nap, sag, tad, pad, and tag might be simultaneously decodable and incomprehensible to an English learner. Even names might not be recognized as such – Cal and Mac aren’t common names. And texts like this are problematic if comprehension is the goal because the “events” in the “story” lack connection and detail. Additionally, the language is stilted and unnatural – this is not how people talk or the way normal books express ideas. The phonics pattern, however, allows for the ELL child to decode accurately, even if they have no idea what any of it means. Consider that any competent adult reader could probably decode passages about open heart surgery or the proper formulation of jet fuel accurately, yet not comprehend most of what they mean.
1 Kavanaugh, K. and Fisher-Ari, T. (2017). Constraints and negotiations: TFA, accountability, and scripted programs in urban schools. The University of Pennsylvania Graduate School of Education’s Online Urban Education Journal. http://www.urbanedjournal.org/volume-14-issue-1-fall-2017-15-years-urban-education-special-anniversary-edition-journal/constraints
AND
Darling-Hammond, L. (2010). The flat world and education: How America’s commitment to equity will determine our future. Teachers College Press.
AND
Ede, A. (2006). Scripted curriculum: Is it a prescription for success? Childhood Education, 121 83(1), 29–32.
2 From an interview with Zaretta Hammond, author of Culturally Responsive Teaching and the Brain: Promoting Authentic Engagement and Rigor Among Culturally and Linguistically Diverse Students. https://www.collaborativeclassroom.org/blog/instructional-equity-with-zaretta-hammond/