Chairman's Corner

Chairman’s Corner: May 2010

May 06, 2010

Social Outcomes: A Revolution in the Making

by Mario Morino, Co-founder of VPP

Years of frustration finally boiled over in January.

I felt compelled to start a series of articles—here and here—on my growing anxiety about the state of efforts in our sector to understand and measure the social outcomes we all hope to achieve.

The Twitter-length summary: Instead of thinking only about HOW to measure, let’s focus on the WHY and WHAT to measure.

And for those of you who, like me, still think in paragraphs: We must not let ourselves overemphasize the mechanics of measurement. We risk losing sight of the most important principles of nonprofit management: Gaining real clarity on the change we are trying to create in the lives of those we serve, figuring out what information is most helpful for determining how we’re doing, and using this information to guide our key decisions and actions. This, in a nutshell, is what I mean when I use the term “managing to outcomes.” It’s the only reliable approach I know for ensuring that we’re doing material, measurable, and sustainable good for those we serve.

Similarly, we must gain greater clarity about the role that evaluation plays in the management of an organization. Many people have come to appreciate the value of experimental and quasi-experimental evaluations, often conducted by third parties, to assess the efficacy of specific programs. But far too few have come to appreciate the value of a different kind of evaluation—the continuous, rigorous collection and use of information to guide the management of the organization. This ongoing, management-oriented evaluation is every bit as important as program evaluation for making deep impact in the areas we care so much about.

In this column, I want to offer two constructive additions. First, I will share some recent experiences that give me the sense that positive change is coming. Second, I’ll offer a concrete framework that nonprofits, specifically top executives and the boards that support them, can use to drive this change in their own organizations.

Innovation from the Periphery

In the midst of my anxiety about our sector’s approach to outcomes, I’ve seen some rays of hope that real change is on the horizon. Even as the large majority of nonprofits and funders in the core of our sector continues to “major on minors,” I’ve gotten a glimpse of some remarkable progress on the periphery of our sector. And the more I’ve dug into this, the more certain I am that this innovation will lead to disruptive, constructive change—literally a paradigm shift—within the core.

It is impossible to predict how quickly change will migrate from the periphery to the core. What I can say is that for some funders and nonprofits, change is likely to be swift—perhaps measured in months. For others, change may take decades, especially for funders stuck in their ways and nonprofits that are woefully under-resourced or don’t have a leader to champion outcomes thinking.

But mark my words—change is coming. Once a critical mass of leaders and those who support them get a taste of the way that some nonprofits are managing to outcomes, they’ll have a hard time going back. They’ll feel like sailors navigating by dead reckoning in a world that now has GPS.

In making these predictions, I claim no clairvoyance. In fact, just about anyone with similar experiences in observing technology adoption and the systems change it enables would come to the same inescapable conclusion. Today I’m seeing a convergence of a) a rather select group of nonprofit leaders hungry for information to help them do what they do better and b) fundamental changes in technology and data architecture and accessibility. This convergence is eerily familiar to geezers like me who worked in the 1970s, ’80s, and ’90s with the likes of Boeing, the U.S. Department of Defense, and Federal Express to implement early versions of performance-management systems. We helped executives peer into the ways that information systems could help them manage their resources and produce improved results (i.e., outcomes). Once executives saw the potential with their own eyes, their view of their world was never the same.

Positive Outliers

For some time, we’ve known there are positive outliers within the nonprofit sector—shining lights like Youth Villages and Nurse Family Partnership that demonstrate the power of information-based management and rigorous, management-oriented evaluation.

At Venture Philanthropy Partners (VPP), we’ve watched the Latin American Youth Center make significant progress in adopting an outcomes orientation, take material steps toward managing to outcomes, and initiate an evaluation approach that could lead to an evidence-based program.  LAYC’s work in outcomes measurement and program evaluation has improved dramatically over the past five years. Today, LAYC is seen as a leader in the nonprofit community in the creation and implementation of data-collection systems, the use of data to evolve program design, and the generation of program-outcome information within a multi-service organization.

Recently, I had an opportunity to participate in demonstrations of two performance-management systems that lifted my sights and hopes even more. Both systems have been recently implemented and in use less than a year. Both help users do what they do better and make what they do easier. Both encourage and reward curiosity and continuous exploration on how to do things better. Both were highly reminiscent of the paradigm-breaking phenomena I witnessed in my business career.

One was the system created by the nonprofit Cleveland Clinic, which I serve as a trustee. I recently had the good fortune to sit down for a meeting with several of the visionaries leading this work.

In brief, they have developed a system that gives administrators and clinicians powerful and easy-to-use tools for making smart administrative and patient-focused healthcare decisions. The system feeds off the data from the Clinic’s repository of electronic medical records and is augmented with an array of other well-thought-through quantitative and qualitative data—from information on patient experience to data on blood utilization. Using this same platform, the Clinic recently started sharing data with a consortium of 117 academic medical centers and their 234 affiliated hospitals.

The other system was equally impressive—given that the organization that built it is not a world-renowned medical institution but rather a network of urban charter schools serving children of low-income families. The organization is Friendship Public Charter School, a VPP investment partner.

In 1998, Friendship Founder Donald Hense and I stood in jeans outside a run-down DC elementary school. He pointed across the street saying, “That’s where we’re going to put our first school.” Today, to Donald’s great credit, Friendship is a thriving network of nine schools and academies, serving 5,697 children.

Friendship’s performance-management system produces dashboards for each student, teacher, classroom, and school—providing timely insights (qualitative and quantitative) on how students are doing on the skills they need to learn. This information, easily available to all teachers and shared with students and their families, allows for much earlier and more effective intervention when kids are having trouble.

Ms. Piccoli is a second-year teacher at one of the Friendship schools. This year her classroom included a majority of students who were low performers relative to their grade-level peers. According to Ms. Piccoli, “I was petrified to show students their data at the beginning of the school year as many were barely readers. I thought it would unsettle the entire class and lead to overwhelming tension and anxiety.” Sharing the data with students, however, is a non-negotiable requirement in Friendship’s model and is expected of all teachers. “My students responded to the data. They helped each other. They knew what they had to do and they kept improving. They have become cheerleaders who encourage each other.” Ms. Piccoli’s students maintain their own graphs, which they color in with their results after each assessment. “I cried when I saw on my last interims how well the students did. It was the first time that they read the assessment themselves rather than having it read to them.” Each of Ms. Piccoli’s students has become a reader. And by taking ownership of their own data, the students have gained confidence in themselves as learners.

At the beginning of this year, Friendship added non-academic indicators related to students’ well-being to its performance system. According to Friendship COO Patricia Brantley, “We saw immediately the interrelationship between struggling teachers and struggling classrooms. Attendance and discipline issues weren’t spread out evenly among classrooms; there was a clear correlation between student non-academic outcomes and teacher performance.”

At Friendship’s first meeting to share the results of attendance and truancy disparities between classrooms, one principal remarked, “Kids can’t just fall through the cracks anymore, because we can see them right when they need us to do so. This is the data that I needed to ensure that every adult is focused on the most important work.” In Brantley’s words, “We use the data as the common driver of urgency for leadership and urgency for management.”

As word gets out about what Friendship has built, it will set a higher bar for schools around the country—including affluent private schools—and give a new sense of what is possible.

I hope that those who have an opportunity to see systems like these will walk away with more than mere appreciation for the technologies they use and the aesthetics of their user interfaces. Far more important is the mindset of the leaders who put these systems in place.

In both cases, members of senior leadership had the vision, passion, conviction, and drive to make the very difficult a reality. Within their own contexts, these leaders embraced a strong performance culture premised on an understanding of the importance and value of managing to outcomes. They saw that being information-informed was essential to achieving their mission.

They wanted to do this not because it’s “important,” not because it’s a trend or a good marketing tool, not because a funder said they had to. They did it because they believed it integral to ensuring material, measurable, and sustainable good for those they serve. In doing so, they exemplified the words of the legendary management expert W. Edwards Deming: “In God we trust. Everyone else brings data.”

One member of our VPP team aptly summed up this way of thinking: “I think it takes a special kind of leader to get this. The fact is, many leaders/staff in the human service nonprofit sector are really ‘people’ people. They aren’t data people, systems people, or tech people. They are in it from a place of empathy, compassion, passion. You have to have both the left and right brain fully wired to get the connection between the two. And you have to be unafraid of what the data might turn up at first. It will show where you are weak. It will show where you have failed.  You have to embrace that and have the full support of your board to go down that path to improvement.”

This mindset and attitude are by far the biggest differentiators between those at the innovative periphery and those in the lagging core.

A Framework to Provoke Introspection

Below, I offer a framework that nonprofit organizations can use if they want to evolve to the practice of managing to outcomes—and funders can use to support nonprofits that are truly motivated to evolve.

This framework is far from perfect, as VPP’s investment partners made clear during a wonderfully open and candid discussion in April. But it reflects many years of implementing management systems in the private sector and more recent experience in the nonprofit sector to assess performance and focus on outcomes. It’s also informed by a wealth of views from people smarter than I, who have been kind enough to share their thinking over the years and who provided wonderful feedback in response to the previous articles in this series.

As you will see below, my starting premise is that it takes a bold spark to ignite outcomes and performance thinking. In an ideal world, this spark would emanate from the board, given that the most important function of a board is ensuring that the organization is aligning its work to its mission—i.e., “mission effectiveness.”

But, of course, reality is rarely neat and orderly. It may be that a visionary executive or manager—either one who is new to the organization or one who has been with the organization for years—steps forward against all the odds and naysayers and takes responsibility for driving toward a greater outcomes focus.

Let me say this as bluntly as I can to nonprofits and funders alike: The challenge of managing to outcomes has little to do with systems, processes, or technology. The real challenge is that organizations cannot hope to manage to outcomes unless they have in place an engaged board; leadership with conviction; clarity of purpose; and a conducive, supportive “performance culture.”



Questions to Guide You

To put this framework to use, I have compiled a set of questions you can use to determine where your organization stands on each factor: Questions to Get You Started

These questions are applicable for most, but not all, nonprofits. They are probably most relevant for nonprofits with annual budgets of $2 million or more (not that budget is the only pertinent gauge). There really is a minimum organizational size necessary to fully take this on, but I don’t want to hand out too many exemptions or “indulgences.” Even small nonprofits should be expected to understand, with at least some level of rigor, what outcomes they would like to achieve for those they serve and how they might begin assess outcomes as they grow larger. And boards should demand this conversation.

The Poets Prophecy

For all the frustrations I have voiced in this series, I do sense we are at, or nearing, one of those inflection points when we will see early innovators at the periphery begin to influence the many nonprofits and funders at the core.

The White House Office of Social Innovation and Civic Participation and the Social Innovation Fund will amplify this trend. The very notion that the Executive Office of the President of the United States is advocating for “improved measurable outcomes and the use of rigorous evidence for grant or investment selection” has the potential to fundamentally reshape the landscape.

In the late 1980s, I asked a friend when he thought technology would truly become pervasive and broadly accepted. His answer was remarkable: “When the poets begin to program.” When the Internet took off in the ’90s, lo and behold this insightful prophecy came true.

So when will managing to outcomes become a broadly accepted part of what it means to operate in the social sector? When the truly mission-driven leaders, whose compassion and caring have been their driving force, find it easy and valuable enough to collect and use data in the natural course of their work.

Eventually, outcomes and metrics will no longer be the exclusive province of a select breed of evaluation and outcomes experts. Instead, outcomes and metrics will be seen for what they truly are—the means by which boards, leadership, and staff can manage nonprofits to be more focused, more effective in delivering programs and services, more efficient in administration, more capable of demonstrating their progress to others, and most important, more conditioned to explore different ways to create greater benefit for those they serve.