Sharpening the Tools of Improvement

We’ve watched enough Grey’s Anatomy and Chicago Hope to know what happens when a patient arrives at a hospital needing medical care – and how doctors are trained to use patient input, examinations, tests, and scans to recommend a treatment plan. We also know that sometimes an initial treatment doesn’t result in recovery – and that doctors don’t wait a year to find out. Patients are monitored, check-ups occur; depending on those results, more information might be needed, or a different medication, or an entirely new approach. What’s more, initial and follow-up treatments are grounded in medical research and best practice; what’s worked for others with similar ailments and needs. If someone presents with an irregular heartbeat, you don’t give them a decongestant and call it cured.

With Improvement Science (IS), education is once again borrowing a page from the medical field, with good reason. Like patients, our students need urgent and expert care from trained professionals who can use all types of data to understand what’s standing in the way of educational progress and apply solutions until barriers are removed. Further, we need to get out of the habit of tackling challenges haphazardly, layering on solutions without clarity on which issue they are meant to solve, let alone if they are successful.

We recently attended the Carnegie Foundation for the Advancement of Teaching’s fourth annual Improvement Science summit to learn from education IS pioneers and join in the conversation about supporting these improvement practices in schools and districts. The summit reinforced some of our lessons from working with educators on IS and gave us some additional food for thought on how leaders can get the most out of this approach.

Improvement Science – What is it?

Grounded in best practices from the health industry, Improvement Science provides a structured method that teams can use to:Part intervention, part research and development, this approach relies on a set of structured protocols—including Ed Demming’s now-famous Plan, Do, Study, Act cycles—and teams of educators and leaders working together to proactively find, understand, and scale solutions.

How is it Different From Other Approaches?

Improvement Science takes a different tack than several other improvement methods. Here’s how we differentiate them:

• Performance Management Systems are primarily used for accountability or tracking longer-term progress. Much of the data collected represent “lagging” indicators, i.e., data that are collected and reviewed on a less-frequent basis to check in on the overall health of a system and can be used to identify challenges but not necessarily solutions. Think annual state test results, graduation rates, and annual check-ups, not acute or urgent care.
• Evidence-Based Practice Models encourage leaders and educators to adopt practices that have been successful elsewhere, as proven by extensive research and/or field trials lasting 3-5 years. While these practices have a role in Improvement Science (change ideas should be things that are expected to actually work!), models relying on evidence from others’ experiences don’t guarantee that they will work in the unique context of every school or district. They can provide a place to start, but they need to be monitored closely to determine whether they are working in the context in which they are being implemented – and why.
• School-based Learning Communities tend to focus on solving for barriers to learning for individual or a limited subset of students. Like IS teams, problem solvers are directly involved in finding solutions; unlike IS teams, school-based learning communities don’t always rely on a structured, action-oriented approach, consider systemic conditions that are impacting a problem, or have a focus on understanding the applicability of a solution to different contexts.

While each of these deserves a place in our improvement practice toolkit, we think IS holds particular promise for a few reasons:

• A focus on systemic impacts to drive lasting change. Rather than blaming an individual, IS pushes leaders to consider systemic conditions contributing to a problem. Before pointing a finger at a principal, for example, let’s look at how that principal was prepared, recruited, selected, and supported in their role. A focus on systems also allows leaders to check for coherence, ensuring that all aspects of a system are aligned and working towards key goals.
• Intentional engagement of a group of leaders and learners who are close to the problem. Multiple, varied perspectives result in a more nuanced view of the problem and more creative and comprehensive solutions. Teams serve an investigative role, each member helping to shed light on aspects of the challenge and bringing forward ideas for addressing it.
• Use of multiple data types to identify root causes. We wrote about some of these sources in an earlier post. Bottom line: the more you understand something, the more effective you are likely to be in solving for it. Multiple data types are a must.
• Specificity of the improvement target and curtailing the wait-and-see approach. IS requires teams to get tight on the improvement they are seeking so they can match solutions appropriately and use rapid cycles (~30-90 days or multiple episodes) to assess impact.
• Not a one and done. Success isn’t an event; it’s a process. There are always more targets to reach.
• Reduces a tendency to layer solutions. Rather than pile on solutions, IS is about finding the ones that work, understanding why, and moving them to scale.

Considerations for Leaders

Our work in IS launched about a decade ago with supports to Kansas leaders—where we leveraged PDSA cycles to help schools and districts move out of low-performing status—and continues today in Hawaii, where we work with principals to introduce them to the tools, practices and mindsets of improvement science. Based on that work—and on conversations with peers similarly using IS to improve educational opportunities and outcomes across the country—here are a few considerations for leaders hoping to put some IS muscle behind their improvement approach:

• Capacity building is a big lift. IS represents a new way of working together and, as we all know from Common Core implementation, practice change is a long game. To get the most out of IS, teams need intensive support to learn and apply the protocols and processes that make it so powerful. In Hawaii, we’ve spent nearly a year helping principals build capacity and confidence to hone in on a problem, collect data, identify levers, and related change ideas. The philanthropic community is providing added resources to support the infrastructure and capacity building needed to give this work legs. We all need to be thinking about the scope and scale of getting this right – and planning accordingly.
• Teaching and learning must remain at center stage. The reason we use IS is to improve student outcomes, and it’s important to frame our targets and solutions with that in mind. Even if we’re focusing on a non-academic target (e.g., chronic absenteeism), we must consider instructional practice as part of a cause and solution. IS encourages us to look inward, and our core business must always be part of that look.
• There’s power in a group. IS harnesses the power of a group to name, investigate, and test change ideas to solve challenges. It’s also about field building, understanding why solutions work and in what contexts and bringing them to scale. For both to be effective, group structures are important. IS uses a structure called Networked Improvement Communities, which allow teams to focus on very specific problems within a larger problem space. The problem focus is chosen after carefully considering root causes and parts of the system impacting it (for example, chronic absenteeism as a problem space with a specific focus on middle school students of color). The more individuals and teams can learn and share with one another, the more likely we will be to bring these practices—and solutions–to scale and better serve students with the precision they deserve.

Recommended Reading

• Learning to Improve: How America’s Schools Can Get Better at Getting Better
• The Improvement Guide: A Practical Approach to Enhancing Organizational Performance
• Carnegie Foundation for the Advancement of Teaching
• Institute for Healthcare Improvement