Three-Year National Study Confirms Effectiveness of Core Knowledge Sequence
by Michael Marshall, Associate Director of Research and Communications,
Core Knowledge Foundation from Common Knowledge, Volume 12, No. 1, Winter 1999
Copyright 1999 by the Core Knowledge Foundation. Not to be copied or reproduced without permission from the Core Knowledge Foundation, 801 E. High Street, Charlottesville, VA 22902.
This article is a synopsis of the 148-page report, National Evaluation of Core Knowledge Sequence Implementation: Final Report by Sam Stringfield, Amanda Datnow, Geoffrey Borman, and Laura Rachuba of the Center for Social Organization of Schools, Johns Hopkins University, 3003 N. Charles Street, Suite 200 Baltimore, Maryland 21218. The full report is available in electronic format only. To receive one, please contact us.
A three-year study of Core Knowledge schools across the country reports that when the Core Knowledge Sequence is really implemented, it really works. Researchers found that students at schools where more than 50 percent of classrooms used the Sequence had higher scores on norm-referenced tests and on criterion-referenced tests of Core Knowledge topics than students at comparison schools. Their report calls the academic gains "educationally meaningful."
The independent study was commissioned in 1995 by the Brown and Walton Foundations to learn the effects of using the Sequence in a variety of school contexts and to determine the conditions under which Core Knowledge is likely to achieve reasonably full implementation.
How It Was Done
The study was conducted by Sam Stringfield, Amanda Datnow, Geoffrey Borman, and other researchers at the Center for Social Organization of Schools at Johns Hopkins University, as well as researchers at the College of Education at University of Memphis. It followed six schools considered promising new Core Knowledge sites and six schools regarded as advanced implementers. All were matched with demographically similar schools in their districts that served as controls. Subject schools were identified in Colorado, Florida, Ohio, Maryland, Tennessee, Texas, and Washington, and reflected various community and socio-economic contexts. Approximately half serve a majority population of students eligible for the federal free lunch program.
At the end of three years, nine of the 12 schools had reached moderate or high levels of implementation. Core implementation improved and increased at four of the six new sites and at five of the six advanced sites. Implementation waned at one new site and at one advanced site, leading the researchers to conclude that although all 12 study schools claimed to be using Core Knowledge, 10 schools "were authentically doing so."
A two-person research team visited each of the schools a total of five times. Visits lasted two to three days. The teams recorded notes of classroom activities and also used an instrument called the Classroom Observation Measure, which has been validated in other studies of elementary classroom instruction. Researchers also surveyed teachers in grades one through five in all 12 schools in 1997. In 1998, only teachers in grades three through five were surveyed as these were the cohorts followed during the period of the study. The survey questionnaire included general questions on Core Knowledge implementation and also asked whether teachers had taught or intended to teach particular core knowledge topics during the school year. An average of 43 percent of teachers returned their surveys in 1997 and 84 percent returned the surveys in 1998.
Researchers examined experimental-control differences in achievement gains over three years, the relationships between level of implementation and academic gains, and differences in gain by cohort (first-through-third-grades and third-through-fifth-grades).
Factors Influencing Successful Implementation
The Johns Hopkins team found that the degree to which Core Knowledge was implemented was a significant predictor of student achievement gain. Researchers concluded that successful implementation of Core Knowledge required leadership from the principal, teacher willingness to change, and support from the district, or at least non-interference from the district. Full implementation was also more likely if money was available for the purchase of Core-related materials and paid teacher planning time, and if schedules, classrooms and professional development programs were arranged to support Core. Implementation was hindered by resistance from principals or teachers, and by strong pressures to comply with state standards and accountability systems that were not aligned with Core.
Since the Core Knowledge Foundation does not prescribe teaching strategies, methods for teaching Core are chosen by local schools and their teachers. According to the report, implementing Core Knowledge "consistently contributed to making instruction more interesting and content rich for students, provided coherence to the curriculum, and contributed to increased teacher collaboration and professionalism. Core Knowledge was also associated with more hands-on, activity-based instruction, and …was associated with greater academic engaged time in schools. One side effect, often viewed negatively, was that planning for Core Knowledge teaching is very work intensive."
Independent work by students was the most prevalent instructional strategy observed "We can infer that much of the independent work was skill-oriented, as sustained writing/composition (either self-directed or on teacher-generated topics) was observed in only 29 percent of classrooms," according to the report. "Direct instruction of the material was also commonly observed. Interdisciplinary instruction characterized almost half of classrooms. Thirty percent of classrooms had some experiential or hands-on learning activities, and cooperative learning was observed in 44 percent of classrooms."
Time spent teaching Core Knowledge varied. Between 60 and 75 percent of teachers at advanced sites said they spent more than 50 percent of their time teaching Core material. Teachers at new sites generally estimated the time they spent teaching Core units at less than 50 percent. The greatest differences in content coverage between the schools appeared to be in third grade mythology, fifth grade literature and fifth grade American civilization topics.
The absence of an official Core Knowledge implementation plan led to "substantial variability over the schools in how they chose to implement the curriculum," the report states. Many teachers at the strongest implementation sites said the lack of specific implementation instructions was "one of the reform's chief positive attributes…. Clearly, it allowed for local variation in organization and implementation strategies, and in diverse contexts unique strategies appeared to be succeeding. Still, there were schools in which a more concrete plan with well-described steps …might have led to a higher level of implementation," the authors conclude. [Before 1996, the Core Knowledge Foundation did not offer or require professional development. There are now more than 130 Core Knowledge trainers working with schools across the country.]
The researchers also investigated the schools' decision to adopt Core. "The first pattern was that none of the educators mentioned Core Knowledge content as a motivating factor," they report. "Rather, educators were seeking sequential, content-rich curriculum and the Core Knowledge Sequence met this demand." Second, in none of the schools did the impetus for teaching Core arise from teachers. However, those schools where educators either participated in choosing the reform implemented with much greater success than schools where teachers felt Core Knowledge was imposed against their will. "What appears to make Core Knowledge different from some other reform efforts is that there appears to be less teacher resistance overall, especially as teachers begin to implement the program. This may be because teachers are generally more resistant to changes in instructional approaches (the "how") than changes in curriculum (the "what")," the report states.
Although most teachers said the time and work required for preparation lessened over time, especially after the first two years, the considerable and even burdensome amount of time required by teachers in planning constituted a hindrance to implementation in some schools. In addition, almost every teacher interviewed expressed difficulty in finding age-appropriate materials for various units.
The absence of prepackaged materials for teachers and students produced a great diversity of teaching strategies. Three techniques used most often were teacher-made units, thematic units and trade books. Particularly at advanced sites, "the majority of teachers spoke positively about the absence of materials," Stringfield and colleagues note. One principal said: "I would hate to see us formalize [Core Knowledge] to a point that it's almost a textbook approach. Because once we start having a Core Knowledge textbook, then Core is going to be just like everything else. It's not going to be a real change process; it's just going to be another series you adopt." Some teachers also agreed that prescribed lesson plans and materials for Core Knowledge would reduce the possibility of positive collaborative relationships among teachers that were created through joint planning. Many teachers also enjoyed researching and developing lesson plans that fit with their own style of teaching. A teacher at an advanced site explained: "I think when you get into how to teach, that's when you meet resistance … . When you bring in a new package that says this is the stuff we want you to teach and this is how to teach it,' I think a wall comes up immediately."
The report summarizes the key factors in implementation as:
- Decision-making autonomy was helpful.
- Arranging common planning time for teachers greatly aided implementation.
- Schools that implemented fastest had grant money to purchase resources.
- District support helped; its absence did not necessarily hurt
- State and district demands related to standards and accountability (more specifically, standardized tests) constrained implementation at most sites.
Qualitative Outcomes
Researchers confirmed that the following predicted benefits "were in fact associated with Core Knowledge implementation":
For students, Core does:
- Provide a broad base of knowledge and a rich vocabulary
- Motivate students to learn and create a strong desire to learn more
- Promote the knowledge necessary for higher learning
For the school, Core does:
- Provide an academic focus and encourage consistency in instruction
- Provide a plan for coherent, sequenced learning from grade to grade
- Promote a community of learners — adults and children
- Become an effective tool for lesson planning and communication among teachers and with parents
-
Guide thoughtful purchases of school resources
Beyond these, the study identified unexpected benefits:
- Core Knowledge created coordination in the curriculum.
- Implementing Core improves the professional lives of teachers. "Core Knowledge was viewed very favorably by teachers and seen as an enhancement to their lives. Overwhelmingly, teachers enthusiastically encouraged their teacher friends to implement Core Knowledge. This is a very important finding."
- Implementing Core Knowledge led to increased teacher collaboration. Such "genuine collaborative work among teachers that has a focus on the curriculum and instruction is all too rare in education," the researchers note.
- Core Knowledge enriched students' classroom experience. "Teachers reported that it was not just certain students who were excited by Core, but all students. … The benefits are great for teaching those children who would normally not be exposed to such subjects at home."
- Core Knowledge challenged conventional assumptions about student ability. "Many teachers reported being initially skeptical that Core Knowledge content was not developmentally appropriate for elementary students. However almost all teachers interviewed found that no matter what students' starting points were — low achieving, average or high achieving — they were able to grasp and gain from learning the Core material."
- Students built on what they learned previously in Core Knowledge. "Teachers find that in fact students make connections to Core topics they learned in previous grades. Students make lasting academic connections because of the integration of the curriculum and [its] spiraling structure."
- Core Knowledge increased students' interest in reading. Teachers report that "students are learning to read bigger words sooner. There's an interest to read and to learn." At a number of schools, "educators cited the fact that students are more interested in reading non-fiction as one of the main benefits of Core Knowledge."
- Core Knowledge increased parent satisfaction. "Parents are thrilled, thrilled, thrilled," according to one teacher, another of whom said, "Our parents are elated with the results of Core." Researchers found "no obvious negative outcomes for students, though teacher planning effort was reported to be 'intensive' and 'tiring.'"
Testing Outcomes
The study analyzed student-level effects, and made three preliminary observations:
First, 10 of 12 Core Knowledge schools were obtaining measures of student engagement in the "highly effective" range.
Second, the two schools with the highest mean student engagement ratings were also schools that had been deemed "highly implementing" and the two schools with the lowest engagement rating were the two schools rated as the lowest implementers.
Third, data indicate that in several of the more highly implementing schools, teachers were able to sustain student interest in each period's academic content. Data suggest that "students find Core content stimulating and would contradict any assertion that students are 'turned off' in schools that strongly implement Core Knowledge."
Only students for whom both pre- and post-testing data was available were included in gain-over-time analyses. One cohort of 1,093 students began the 1995-96 school year as first grade students, and the other cohort of 1,011 students began the same year as third grade students.
The researchers administered two subtests from the Comprehensive Test of Basic Skills, Fourth Edition (CTBS/4), to the cohorts at three stages. They derived Normal Curve Equivalent Scores (NCEs) from the CTBS/4 Math Concepts and Applications subtest and the Reading Comprehension subtest.
In spring 1998, data collection on reading and math achievement yielded CTBS/4 data on 663 students, or 61 percent, of the 1,093-cohort baseline sample, and on 653, or 65 percent, of the 1,011-cohort baseline sample. When low and high implementing sites taken together, the effect of Core Knowledge on reading and math achievement was not statistically significant. However, on further analysis, the effects on normed tests became statistically significant when schools with moderate to high implementation were contrasted with low-implementing sites as controls. The Johns Hopkins statistician reports that the the gain difference on standardized tests between low and high implementing schools varied from 8.83 NCEs to 16.28 NCEs. That is an average rise of about 12 NCEs (similar to percentile points) over the controls, more than half a standard deviation — a very significant gain.
The researchers give the following explanation of the two last figures in the following tables: "One figure is the relationship between a one-quartile increase in level of implementation (i.e., 25% more classrooms reliably implementing Core). The other represents the expected difference between the lowest-implementing Core school and the highest-implementing Core school in our samples. The NCE metric is very similar to the percentile metric".
Core Knowledge Effect Sizes by Test for Schools with Implementation Rates Greater than 50%.
First-through-Third-Grade-Cohort | Third-through-Fifth-Grade-Cohort |
|---|---|
 |
 |
| Grade 1–3 | Grade 3–5 | |
|---|---|---|
| Math | Coefficient = .121 Quartile increase Low vs. High implementation |
Coefficient = .223 Quartile increase Low vs. High implementation |
| Reading | Coefficient = .199 Quartile increase Low vs. High implementation |
Coefficient = .143 Quartile increase Low vs. High implementation |
The researchers also created Core Knowledge Achievement Tests in science, language arts, and geography/world civilization/American civilization. Naturally, on these tests Core schools outperformed their "comparison" schools where Core was not being taught. But the test results show that students retained the Core Knowledge content they were taught, and, since this knowledge is carefully chosen and cumulative, what they learned is predicted (by E. D. Hirsch, Jr.) to enhance students' vocabulary, reading skill, and learning ability in later grades. The tests were given to all third and fifth graders in the study. Each test had 20 multiple-choice questions; the history and geography test had one item needing a written answer. Statistically significant, "educationally meaningful," achievement gain was found in every subject for both cohorts.
In sum, when scores are analyzed according to the degree that a school has implemented Core, data show academic improvement was accelerated at sites that were implementing strongly. As the argument for Core Knowledge predicts, the growth of the general knowledge base showed cumulative effects. "The third-through-fifth grade results contrast the findings for first graders, in that higher levels of Core Knowledge implementation in the later elementary grades appear to have a more profound influence on students'" reading and math tests, the report confirms. "If this trend, predicted by theory, were to continue in later grades, the gains in reading comprehension would accelerate — a subject to be investigated further."
Effect Size by Level of Implementation for the First-through-Third-Grade Cohort
Effect Size by Level of Implementation for the Third-through-Fifth-Grade Cohort
Why the Positive Effects?
Core Knowledge implementation "produced clarity of goals, less repetitiveness in the curriculum and more content rich instruction for students," the report concludes. "It might be inferred that the better relative performance by the later Core Knowledge cohort could be explained by the cumulative effect of a content-focused curriculum on general academic skills. Since normed tests are not tied to a particular sequence, the cumulative effects of carefully sequenced content would be more likely to exhibit themselves in the later grades, as a gradual result over time."
Likewise, the schools' improved implementation during the period of the study also showed a cumulative effect. "The correlation between level of implementation and effect size indicates that when schools implemented the Core Knowledge Sequence with greater reliability and consistency, students achieved improved scores on all tests. Considering only those schools in which the research staff observed Core Knowledge curriculum and instruction in more than 50 percent of classrooms, one sees marked increases in the effect size favoring Core Knowledge. Among first-through-third grade students, improved implementation was related to substantially higher Core Knowledge test outcomes. The results for third-through-fifth-grade students suggest that higher levels of implementation were associated with larger, educationally meaningful effects on the norm-referenced test, and on the Core Knowledge tests."
What appears to have mattered most, the report suggests, was "the fact that the curriculum was specified, and less so that it was the Core Knowledge content. This led us to the conclusion that the benefits associated with a specific curriculum may not be limited to Core Knowledge per se, but instead may be applicable to other specified curricula, even a fully articulated curricular sequence developed by schools themselves — so long as the content covered is broad, sequential and well-grounded in theory and research."
 |
 |
 |
|
||
 |
|
 |
|
||
Last updated: Fri, May 23 2008
