Skip to:

Education Reform

  • Evaluating The Results Of New Teacher Evaluation Systems

    Written on March 24, 2016

    A new working paper by researchers Matthew Kraft and Allison Gilmour presents a useful summary of teacher evaluation results in 19 states, all of which designed and implemented new evaluation systems at some point over the past five years. As with previous evaluation results, the headline result of this paper is that only a small proportion of teachers (2-5 percent) were given the low, “below proficiency” ratings under the new systems, and the vast majority of teachers continue to be rated as satisfactory or better.

    Kraft and Gilmour present their results in the context of the “Widget Effect,” a well-known 2009 report by the New Teacher Project showing that the overwhelming majority of teachers in the 12 districts for which they had data received “satisfactory” ratings. The more recent results from Kraft and Gilmour indicate that this hasn’t changed much due to the adoption of new evaluation systems, or, at least, not enough to satisfy some policymakers and commentators who read the paper.

    The paper also presents a set of findings from surveys of and interviews with observers (e.g., principals). These are in many respects more interesting and important results from a research and policy perspective, but let’s nevertheless focus a bit on the findings on the distribution of teachers across rating categories, as they caused a bit of a stir. I have several comments to make about them, but will concentrate on three in particular (all of which, by the way, pertain not to the paper’s discussion, which is cautious and thorough, but rather to some of the reaction to it in our education policy discourse).

    READ MORE
  • Teachers And Professional Collaboration: How Sweden Has Become The ABBA Of Educational Change

    Written on March 2, 2016

    Our guest author today is Andy Hargreaves, the Brennan Chair in Education at Boston College. He is the coauthor of Professional Capital: Transforming Teaching in Every School, which won the 2015 Grawemeyer Award for the idea in education most likely to have the most effect on practice worldwide. He is also the 2016 recipient of the Horace Mann League’s Outstanding Friend of Public Education Award. An extended version of this column originally appeared in Pedagogiska Magasinet, the Swedish teachers’ magazine in February 2016.

    In the 1960s and 70s, Sweden’s economic productivity and social engineering were the envy of democrats all over the world. The nation’s comprehensive schools were an inspiration for public education reformers in the United Kingdom and many other nations too. In Sweden, market prosperity and the collective good went side by side. It was a country where, like the nations’ classic pop group, Abba, people banded and bonded together really well.

    In the 90s, however, Sweden entered an age of what political scientists call free-market neo-liberalism, and educational reform was at the leading edge of it. In some ways moving ahead of the US trend, Sweden introduced large numbers of competitive “free schools”, funded with public money but no longer regulated by their school districts. Hedge fund companies were the largest single group of owners of these schools. Sweden’s society and its schools were, in the titles of two of Abba’s songs, now driven by a “Winner Takes it All” culture of “Money, Money, Money!” Between 2003 and 2012, Sweden experienced the greatest deterioration in PISA scores out of all OECD countries who were performing above average in 2003. Despite the country's proud and internationally admired egalitarian tradition, its achievement gaps have been widening faster than in any other country.

    READ MORE
  • Student Sorting And Teacher Classroom Observations

    Written on February 25, 2016

    Although value added and other growth models tend to be the focus of debates surrounding new teacher evaluation systems, the widely known but frequently unacknowledged reality is that most teachers don’t teach in the tested grades and subjects, and won’t even receive these test-based scores. The quality and impact of the new systems therefore will depend heavily upon the quality and impact of other measures, primarily classroom observations.

    These systems have been in use for decades, and yet, until recently, relatively little is known about their properties, such as their association with student and teacher characteristics, and there are, as yet, only a handful of studies of their impact on teachers’ performance (e.g., Taylor and Tyler 2012). The Measures of Effective Teaching (MET) Project, conducted a few years ago, was a huge step forward in this area, though at the time it was perhaps underappreciated the degree to which MET’s contribution was not just in the (very important) reports it produced, but also in its having collected an extensive dataset for researchers to use going forward. A new paper, just published in Educational Evaluation and Policy Analysis, is among the many analyses that have and will use MET data to address important questions surrounding teacher evaluation.

    The authors, Rachel Garrett and Matthew Steinberg, look at classroom observation scores, specifically those from Charlotte Danielson’s widely employed Framework for Teaching (FFT) protocol. These results are yet another example of how observation scores share most of the widely-cited (statistical) criticisms of value added scores, most notably their sensitivity to which students are assigned to teachers.

    READ MORE
  • The IMPACT Of Teacher Turnover In DCPS

    Written on February 3, 2016

    Teacher turnover has long been a flashpoint in education policy, yet these debates are rife with complications. For example, it is often implied that turnover is a “bad thing,” even though some turnover, as when low-performing teachers leave, can be beneficial, whereas some retention, as when low-performing teachers stay, can be harmful. The impact of turnover also depends heavily on other factors, such as the pool of candidates available to serve as replacements, and how disruptive turnover is to the teachers who are retained.

    The recent widespread reform of teacher evaluation systems has made the turnover issue, never far below the surface, even more salient in recent years. Critics contend that the new evaluations, particularly their use of test-based productivity measures, will cause teachers to flee the profession. Supporters, on the other hand, are in a sense hoping for this outcome, as they anticipate that, under the new systems, voluntary and involuntary separations will serve to improve the quality of the teacher workforce.

    A new working paper takes a close look the impact of teacher turnover under what is perhaps the most controversial teacher evaluation system in the nation – that used in the District of Columbia Public Schools (DCPS). It's a very strong analysis that speaks directly to policy in a manner that does not fit well into the tribal structure of education debates today.

    READ MORE
  • The Narrative Of School Failure And Why We Must Pay Attention To Segregation In Educational Policy

    Written on January 28, 2016

    Our guest authors today are Kara S. Finnigan, Associate Professor at the Warner School of Education of the University of Rochester, and Jennifer Jellison Holme, Associate Professor at the University of Texas at Austin.  Finnigan and Holme have published several articles and briefs on the issue of school integration including articles in press in Teachers College Record and Educational Law and Policy Review as well as a research brief for the National Coalition on School Diversity. This is the first of a two-part blog series on this topic.

    Imagine that you wake up one morning with a dull pain in your tooth. You take ibuprofen, apply an ice pack, and try to continue as if things are normal.  But as the pain continues to grow over the next few days, you realize that deep down there is a problem – and you are reminded of this every so often when you bite down and feel a shooting pain.  Eventually, you can’t take it any longer and get an x-ray at the dentist’s office, only to find out that what was originally a small problem has spread throughout the whole tooth and you need a root canal.  Now you wish you hadn’t waited so long.

    Why are we talking about a root canal in a blog post about education? As we thought about how to convey the way we see the situation with low-performing schools, this analogy seemed to capture our point. Most of us can relate to what happens when we overlook a problem with our teeth, and yet we don’t pay attention to what can happen when we overlook the underlying problems that affect educational systems.

    In this blog post, we argue that school segregation by race and poverty is one of the underlying causes of school failure, and that it has been largely overlooked in federal and state educational policy in recent decades.

    READ MORE
  • Evidence From A Teacher Evaluation Pilot Program In Chicago

    Written on December 4, 2015

    The majority of U.S. states have adopted new teacher evaluation systems over the past 5-10 years. Although these new systems remain among the most contentious issues in education policy today, there is still only minimal evidence on their impact on student performance or other outcomes. This is largely because good research takes time.

    A new article, published in the journal Education Finance and Policy, is among the handful of analyses examining the preliminary impact of teacher evaluation systems. The researchers, Matthew Steinberg and Lauren Sartain, take a look at the Excellence in Teaching Project (EITP), a pilot program carried out in Chicago Public Schools starting in the 2008-09 school year. A total of 44 elementary schools participated in EITP in the first year (cohort 1), while an additional 49 schools (cohort 2) implemented the new evaluation systems the following year (2009-10). Participating schools were randomly selected, which permits researchers to gauge the impact of the evaluations experimentally.

    The results of this study are important in themselves, and they also suggest some more general points about new teacher evaluations and the building body of evidence surrounding them.

    READ MORE
  • Who Are (And Should Be) The Teaching Experts?

    Written on November 19, 2015

    Our guest author today is Bryan Mascio, who taught for over ten years in New Hampshire, primarily working with students who had been unsuccessful in traditional school settings. Bryan is now a doctoral student at the Harvard Graduate School of Education, where he conducts research on the cognitive aspects of teaching, and works with schools to support teachers in improving relationships with their students.

    How do we fix teaching?  This question is on the mind of many reformers, researchers, politicians, and parents.  Every expert has their own view of the problem, their own perspective on what success should look like, and their own solutions to offer.  The plethora of op-eds, reports, articles, and memoranda, can be mindboggling.  It is important to take a step back and see whether we all even consider teaching expertise to be the same thing.  Just as importantly, where does, and should, it reside?

    In a New York Times op-ed, “Teachers Aren’t Dumb”, Dr. Daniel Willingham explains that teachers aren’t the problem – it’s just how they are trained. As a teacher, I appreciate a respected person from outside of the profession coming to our defense, and I do agree that we need to take a hard look at teacher preparation programs.  I worry, though, that a call to focus more on the “nuts and bolts” of teaching – in contrast to the current emphasis on educational philosophy and theories of development – could create an alarming pendulum swing.

    This recommendation is a common message, promoted both by those in academic research as well as fast-tracked teacher preparation programs.  It sees academics and researchers as the generators and holders of the most important expertise and asks them to then give direction to teachers.  By mistaking different kinds of expertise, it inadvertently lays a path towards teachers as technicians, rather than true professionals.

    READ MORE
  • Student Discipline, Race And Eva Moskowitz’s Success Academy Charter Schools

    Written on October 19, 2015

    At a recent press conference, Success Academy Charter Schools CEO Eva Moskowitz addressed the issue of student discipline. “It is horrifying,” she told reporters, that critics of her charter schools’ high suspension rates don’t realize “that five-year-olds do some pretty violent things.” Moskowitz then pivoted to her displeasure with student discipline in New York City (NYC) public schools, asserting that disorder and disrespect have become rampant.

    This is not the first time Moskowitz has taken aim at the city’s student discipline policies. Last spring, she used the editorial pages of the Wall Street Journal to criticize the efforts of Mayor Bill De Blasio and the NYC Department of Education to reform the student code of conduct and schools’ disciplinary procedures. Indeed, caustic commentary on student behavior and public school policy has become something of a trademark for Moskowitz.

    The National Move to Reform Student Discipline Practices

    To understand why, it is important to provide some context. The New York City public school policies that Moskowitz derides are part of a national reform effort, inspired by a body of research showing that overly punitive disciplinary policies are ineffective and discriminatory. Based on this research evidence, the American Academy of Pediatrics, the American Psychological Association and School Discipline Consensus Project of the Council of State Governments have all gone on record on the harmful effects of employing such policies. The U.S. Education Department, the U.S. Justice Department, civil rights and civil liberties organizations, consortia of researchers, national foundations, and the Dignity in Schools advocacy coalition have all examined the state of student discipline in America’s schools in light of this research.1

    READ MORE
  • Recent Evidence On The New Orleans School Reforms

    Written on September 30, 2015

    A new study of New Orleans (NOLA) schools since Katrina, published by the Education Research Alliance (ERA), has caused a predictable stir in education circles (the results are discussed in broader strokes in this EdNext article, while the full paper is forthcoming). The study’s authors, Doug Harris and Matthew Larsen, compare testing outcomes before and after the hurricanes that hit the Gulf Coast in 2005, in districts that were affected by those storms. The basic idea, put simply, is to compare NOLA schools to those in other storm-affected districts, in order to assess the general impact of the drastic educational change undertaken in NOLA, using the other schools/districts as a kind of control group.

    The results, in brief, indicate that: 1) aggregate testing results after the storms rose more quickly in NOLA vis-à-vis the comparison districts, with the difference in 2012 being equivalent to roughly 15 percentile points ; 2) there was, however, little discernible difference in the trajectories of NOLA students who returned after the storm and their peers in other storm-affected districts (though this latter group could only be followed for a short period, all of which occurred during these cohorts' middle school years). Harris and Larsen also address potential confounding factors, including population change and trauma, finding little or no evidence that these factors generate bias in their results.

    The response to this study included the typical of mix of thoughtful, measured commentary and reactionary advocacy (from both “sides”). And, at this point, so much has been said and written about the study, and about New Orleans schools in general, that I am hesitant to join the chorus (I would recommend in particular this op-ed by Doug Harris, as well as his presentation at our recent event on New Orleans).

    READ MORE
  • The Magic Of Multiple Measures

    Written on August 6, 2015

    Our guest author today is Cara Jackson, Assistant Director of Research and Evaluation at the Urban Teacher Center.

    Teacher evaluation has become a contentious issue in U.S.  Some observers see the primary purpose of these reforms as the identification and removal of ineffective teachers; the popular media as well as politicians and education reform advocates have all played a role in the framing of teacher evaluation as such.  But, while removal of ineffective teachers was a criterion under Race to the Top, so too was the creation of evaluation systems to be used for teacher development and support.

    I think most people would agree that teacher development and improvement should be the primary purpose, as argued here.  Some empirical evidence supports the efficacy of evaluation for this purpose (see here).  And given the sheer number of teachers we need, declining enrollment in teacher preparation programs, and the difficulty disadvantaged schools have retaining teachers, school principals are probably none too enthusiastic about dismissing teachers, as discussed here.

    Of course, to achieve the ambitious goal of improving teaching practice, an evaluation system must be implemented well.  Fans of Harry Potter might remember when Dolores Umbridge from the Ministry of Magic takes over as High Inquisitor at Hogwarts and conducted “inspections” of Hogwart’s teachers in Book 5 of J.K. Rowling’s series.  These inspections pretty much demonstrate how not to approach classroom observations: she dictates the timing, fails to provide any of indication of what aspects of teaching practice she will be evaluating, interrupts lessons with pointed questions and comments, and evidently does no pre- or post-conferencing with the teachers. 

    READ MORE

Pages

Subscribe to Education Reform

DISCLAIMER

This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from shankerblog.org. The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the Shanker Blog may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.