Skip to:

Education Policy

  • Actual Growth Measures Make A Big Difference When Measuring Growth

    Written on February 25, 2015

    As a frequent critic of how states and districts present and interpret their annual testing results, I am also obliged (and indeed quite happy) to note when there is progress.

    Recently, I happened to be browsing through New York City’s presentation of their 2014 testing results, and to my great surprise, on slide number four, I found proficiency rate changes between 2013 and 2014 among students who were in the sample in both years (which they call “matched changes”). As it turns out, last year, for the first time, New York State as a whole began publishing these "matched" year-to-year proficiency rate changes for all schools and districts. This is an excellent policy. As we’ve discussed here many times, NCLB-style proficiency rate changes, which compare overall rates of all students, many of whom are only in the tested sample in one of the years, are usually portrayed as “growth” or “progress.” They are not. They compare different groups of students, and, as we’ll see, this can have a substantial impact on the conclusions one reaches from the data. Limiting the sample to students who were tested in both years, though not perfect, at least permits one to measure actual growth per se, and provides a much better idea of whether students are progressing over time.

    This is an encouraging sign that New York State is taking steps to improve the quality and interpretation of their testing data. And, just to prove that no good deed goes unpunished, let’s see what we can learn using the new “matched” data – specifically, by seeing how often the matched (longitudinal) and unmatched (cross-sectional) changes lead to different conclusions about student “growth” in schools.

    READ MORE
  • Sample Size And Volatility In School Accountability Systems

    Written on February 24, 2015

    It is generally well-known that sample size has an important effect on measurement and, therefore, incentives in test-based school accountability systems.

    Within a given class or school, for example, there may be students who are sick on testing day, or get distracted by a noisy peer, or just have a bad day. Larger samples attenuate the degree to which unusual results among individual students (or classes) can influence results overall. In addition, schools draw their students from a population (e.g., a neighborhood). Even if the characteristics of the neighborhood from which the students come stay relatively stable, the pool of students entering the school (or tested sample) can vary substantially from one year to the next, particularly when that pool is small.

    Classes and schools tend to be quite small, and test scores vary far more between- than within-student (i.e., over time). As a result, testing results often exhibit a great deal of nonpersistent variation (Kane and Staiger 2002). In other words, much of the differences in test scores between schools, and over time, is fleeting, and this problem is particularly pronounced in smaller schools. One very simple, though not original, way to illustrate this relationship is to compare the results for smaller and larger schools.

    READ MORE
  • The Debate And Evidence On The Impact Of NCLB

    Written on February 17, 2015

    There is currently a flurry of debate focused on the question of whether “NCLB worked.” This question, which surfaces regularly in the education field, is particularly salient in recent weeks, as Congress holds hearings on reauthorizing the law.

    Any time there is a spell of “did NCLB work?” activity, one can hear and read numerous attempts to use simple NAEP changes in order to assess its impact. Individuals and organizations, including both supporters and detractors of the law, attempt to make their cases by presenting trends in scores, parsing subgroups estimates, and so on. These efforts, though typically well-intentioned, do not, of course, tell us much of anything about the law’s impact. One can use simple, unadjusted NAEP changes to prove or disprove any policy argument. And the reason is that they are not valid evidence of an intervention's effects. There’s more to policy analysis than subtraction.

    But it’s not just the inappropriate use of evidence that makes these “did NCLB work?” debates frustrating and, often, unproductive. It is also the fact that NCLB really cannot be judged in simple, binary terms. It is a complex, national policy with considerable inter-state variation in design/implementation and various types of effects, intended and unintended. This is not a situation that lends itself to clear cut yes/no answers to the “did it work?” question.

    READ MORE
  • The Increasing Academic Ability Of New York Teachers

    Written on February 12, 2015

    For many years now, a common talking point in education circles has been that U.S. public school teachers are disproportionately drawn from the “bottom third” of college graduates, and that we have to “attract better candidates” in order to improve the distribution of teacher quality. We discussed the basis for this “bottom third” claim in this post, and I will not repeat the points here, except to summarize that “bottom third” teachers (based on SAT/ACT scores) were indeed somewhat overrepresented nationally, although the magnitudes of such differences vary by cohort and other characteristics.

    A very recent article in the journal Educational Researcher addresses this issue head-on (a full working version of the article is available here). It is written by Hamilton Lankford, Susanna Loeb, Andrew McEachin, Luke Miller and James Wyckoff. The authors analyze SAT scores of New York State teachers over a 25 year period (between 1985 and 2009). Their main finding is that these SAT scores, after a long term decline, improved between 2000 and 2009 among all certified teachers, with the increases being especially large among incoming (new) teachers, and among teachers in high-poverty schools. For example, the proportion of incoming New York teachers whose SAT scores were in the top third has increased over 10 percentage points, while the proportion with scores in the bottom third has decreased by a similar amount (these figures define “top third” and “bottom third” in terms of New York State public school students who took the SAT between 1979 and 2008).

    This is an important study that bears heavily on the current debate over improving the teacher labor supply, and there are few important points about it worth discussing briefly.

    READ MORE
  • Resources On The Social Side Of Education Reform

    Written on January 12, 2015

    Updates to this post will be posted here

    For the past few months, we have been insisting, through this blog series, on the idea that education reform has a social dimension or level that often is overlooked in mainstream debate and policy. Under this broad theme, we've covered diverse issues ranging from how teachers' social capital can increase their human capital to how personnel churn can undermine reform efforts, or how too much individual talent can impede a team's overall performance. This collection of issues may prompt a number of important questions: What exactly is the "social side?" What are its key ideas? I would like to offer a few initial thoughts and share some resources that I've compiled.

    The social side is primarily a lens that brings into focus a critical oversight in the public debate on educational reform and its policies: The idea that teaching and learning are not solo but rather social endeavors that are achieved in the context of the school organization, and within the districts where schools are embedded, through relationships and teamwork, rather than competition and a focus on individual prowess. 

    This social side perspective does a few things:

    READ MORE
  • Constitution For Effective School Governance

    Written on December 10, 2014

    Our guest author today is Kenneth Frank, professor in Measurement and Quantitative Methods at the Department of Counseling, Educational Psychology and Special Education at Michigan State University.

    Maybe it’s because I grew up in Michigan, but when I think of how to improve schools, I think about the “Magic Johnson effect." During his time at Michigan State, Earvin “Magic” Johnson scored an average of 17 points per game. Good, but many others have had higher averages. Yet, I would want Magic Johnson on my team because he made everyone around him better. Similarly, the best teachers may be those that make everyone around them better.  This way of thinking is not currently the focus of many current educational reforms, which draw on individual competition and market metaphors.

    So how can we leverage the Magic Johnson effect to make schools better? We have to think of ways that teachers can work together. This might be in terms of co-teaching, sharing materials, or taking the time to engage one another in honest professional dialogues. There is considerable evidence that teachers who can draw on the expertise of colleagues are better able to implement new practices. There is also evidence that when there is an atmosphere of trust teachers can engage in honest dialogues that can improve teaching practices and student achievement (e.g., Bryk and Schneider, 2002).

    READ MORE
  • Feeling Socially Connected Fuels Intrinsic Motivation And Engagement

    Written on November 20, 2014

    Our "social side of education reform" series has emphasized that teaching is a cooperative endeavor, and as such is deeply influenced by the quality of a school's social environment -- i.e., trusting relationships, teamwork and cooperation. But what about learning? To what extent are dispositions such as motivation, persistence and engagement mediated by relationships and the social-relational context?

    This is, of course, a very complex question, which can't be addressed comprehensively here. But I would like to discuss three papers that provide some important answers. In terms of our "social side" theme, the studies I will highlight suggest that efforts to improve learning should include and leverage social-relational processes, such as how learners perceive (and relate to) -- how they think they fit into -- their social contexts. Finally, this research, particularly the last paper, suggests that translating this knowledge into policy may be less about top down, prescriptive regulations and more about what Stanford psychologist Gregory M. Walton has called "wise interventions" -- i.e., small but precise strategies that target recursive processes (more below).

    The first paper, by Lucas P. Butler and Gregory M. Walton (2013), describes the results of two experiments testing whether the perceived collaborative nature of an activity that was done individually would cause greater enjoyment of and persistence on that activity among preschoolers.

    READ MORE
  • Librarians, Libraries, Serendipity And Passion

    Written on November 6, 2014

    Our guest author today is Connie Williams, a National Board Certified Teacher librarian at Petaluma High School in Petaluma, CA, past president of the California School Library Association, and co-developer of the librarian and teacher 2.0 classroom tutorials.

    Down the road from where I live, on the first-of-the month, a group of vintage car owners gather for a “cars and coffee” meet up. The cars that show up with their drivers cover many years and obsessions. Drivers park, open up the car hoods and take a few steps back and begin talking with other car owners and visitors who happen by. These are people who are interested in the way cars work, their history, and they all have stories to share.

    How do they know so much about their cars? They work on them – gaining insight by hands-on practice and consultations with experts. If they’re wealthy enough, they pay someone else to do the work, yet they don’t just hand over their cars to them. They read about them, participate in on-line groups, ask for guidance, and they drive them. Most often, when they drive them, someone stops and asks questions about their cars and they teach what they know to others. 

    This is an example of the kind of learning we would hope for, for all our students – a passion that is ignited and turns into knowledge that is grown, developed, and shared. In this sense, it is inquiry – asking questions and taking the required steps to answer them – that is at the heart of learning.

    READ MORE
  • The Equity Projection

    Written on October 30, 2014

    A new Mathematica report examines the test-based impact of The Equity Project (TEP), a New York City charter school serving grades 5-8. TEP opened up for the 2009-10 school year, receiving national attention mostly due to one unusual policy: They paid teachers $125,000 per year, regardless of experience and education, in addition to annual bonuses (up to $25,000) for returning teachers. TEP largely makes up for these unusually high salary costs by minimizing the number of administrators and maintaining larger class sizes.

    As is typical of Mathematica, the TEP analysis is thorough and well-done. The school's students' performance is compared to that of similar peers with a comparable probability of enrolling in TEP, as identified with propensity scores. In general, the study’s results were quite positive. Although there were statistically discernible negative impacts of attendance for TEP’s first cohort of students during their first two years, the cumulative estimated test-based impact was significant, positive and educationally meaningful after three and four years of attendance. As always, the estimated effect was stronger in math than in reading (estimated effect sizes for the former were very large in magnitude). The Mathematica researchers also present analyses on student attrition, which did not appear to bias the estimates substantially, and they also show that their primary results are robust when using alternative specifications (e.g., different matching techniques, score transformations, etc.).

    Now we get to the tricky questions about these results: What caused them and what can be learned as a result? That’s the big issue with charter analyses in general (and with research on many other interventions): One can almost never separate the “why” from the “what” with any degree of confidence. And TEP, with its "flagship policy" of high teacher salaries, which might appeal to all "sides" in the education policy debate, provides an interesting example in this respect.

    READ MORE
  • Where Al Shanker Stood: Quick Fixes In Education

    Written on October 21, 2014

    In this "Where We Stand" column, which was printed in the New York Times on March 27, 1983, Al Shanker quotes historian Paul Gagnon to argue that we need to think long-term about the purposes of public schooling and agree on a carefully chosen set of education reform priorities. Failing this, they warn, the U.S. will forever be caught in a churn of futile, quick-fix reform initiatives.

    It never fails. Whenever there's an educational problem, there's always an attempt to solve it with a quick fix. The current problem - the shortage of science and math teachers - is no exception. A quick fix just won't work. Of course, there are a few things that can be done to ease the problem. The most promising short-run idea is to encourage teachers already teaching in other fields but who have a good background in math and science to switch.

    But we won't solve the problem until we know why we have one. It is not just that private industry pays more. It's that there aren't enough students graduating from college in these fields to satisfy the needs of business and the teaching profession. Most students stay away from math and science in college because they didn't get enough of a background in high school. Why? Because math and science course are more difficult than many electives, and most high school students, given a choice between tough courses and easy ones, choose the latter. And it doesn't start there. It goes back to elementary school, and not just with respect to math and science but with the ability to read problems and think them through ... willingness to discipline oneself, to work long and hard.

    READ MORE

Pages

Subscribe to Education Policy

DISCLAIMER

This web site and the information contained herein are provided as a service to those who are interested in the work of the Albert Shanker Institute (ASI). ASI makes no warranties, either express or implied, concerning the information contained on or linked from shankerblog.org. The visitor uses the information provided herein at his/her own risk. ASI, its officers, board members, agents, and employees specifically disclaim any and all liability from damages which may result from the utilization of the information provided herein. The content in the Shanker Blog may not necessarily reflect the views or official policy positions of ASI or any related entity or organization.