by Kenn Oke
A method for developing a measurement instrument that can guide leaders to define the end goal and that provides church planters with a graphic picture of progress.
A FEW YEARS AGO I was part of a breakout group at a church-planting roundtable where we discussed the question, “What is church?” The group was comprised of international and regional directors of church-planting organizations. About fifteen minutes into the discussion it became apparent that very few of the leaders had a working definition of church that was common to their entire organization. Taken together, these leaders represented hundreds of church planters.
I began to wonder how church planters could be sent to the field without a clear concept of what they are commissioned to do. Would that be acceptable in any other setting?
How successful would car manufacturers be if their leaders told factory workers, “Make cars!” and did not provide them with detailed specifications of what they were to build? Absurd! Yet it seemed like that was exactly what many church-planting organizations had done.
When church planters don’t have a working definition of church, they are left with important questions they can’t answer:
How do they know when they’ve finished the job?
How do they give credible progress reports to supporters when there is no clear definition of what they are progressing toward?
How do they know that what they are doing today is getting them to the goal?
How do they decide where best to use their resources?
Furthermore, from an organizational perspective, if leaders have not defined the end goal clearly, can they truly know whether the day-to-day activities of their church planters are actually fulfilling the organization’s mission?
This article presents a method for developing a measurement instrument that can guide leaders to define the end goal (i.e., “church”) and that provides church planters with a graphic picture of progress toward their goal. The instrument informs church-planting strategy by showing church planters what they have accomplished and what is left to be done. Consequently, it creates alignment between the organization’s mission and the day-to-day activities of its church planters. Before getting into the details of developing the tool, however, it will be helpful to review some of the methods of measurement that have been applied to church planting.
The Problem of Measurement Inversion
Church planters who want to measure success often fall into measurement inversion (i.e., they measure what is easy to measure rather than what is most important). Measurement inversion in church planting has been evident in at least two ways.
First, church planters have focused on activities, which are easy to monitor, at the expense of measuring outcomes of the activities. Here is an example taken from a church-planting manual that has been used in at least twenty-seven countries. The manual wisely instructs church planters to set goals that are measurable. The sample goal is, “By the end of June, I will prepare and lead a series of three inductive Bible studies in my cell group on the theme of ‘The Great Commission and Church Planting in Our City’” (Deyneka 1999, 157). The manual suggests that this is measurable in that by June the church planter will be able to say, “[Yes/no] I led the studies.”
Clearly, that is not the important piece of information. It focuses on monitoring the church planter’s activities rather than measuring the success of the studies. A true outcome metric would be, “The people in my cell group have adopted a Great Commission mindset and desire to plant churches in our city.” A similar example comes from Scott Breslin’s article in the October 2007 issue of EMQ. He presents five indicators to track progress in a church plant. The first three, “making contacts,” “sowing”, and “watering”, focus completely on church planters’ activity. The fourth, “reaping”, tracks an undesired outcome in that it counts people who have made a decision but are not connected to the church. Only “keeping” is a true outcome (2007). The tool does not distinguish between activity and outcomes.
Second, measurement inversion also appears in much church growth literature. Although church growth writers repeatedly stress the importance of growing quality churches, they focus their measurements almost exclusively on the size and growth rate of the church. If quality is important, it needs to be measured.
The object of our measurement, therefore, should be the quality or health of the church, not simply its size and growth rate (Warren 1995, 98; Walker 2005, 3-13). This is not to say the church must grow healthy first and then focus on outreach and numerical growth; rather, it means that healthy churches will naturally grow.
Defining and Measuring “Healthy Church”
The best definition of a healthy church is God’s definition! Gary Corwin argues that church planters need to take their focus off of repeating past forms of church, or even creating new forms of church. They need to “recognize essential biblical patterns” and seek to develop these characteristics in their churches (2005, 143). God has defined “healthy church” in his word, and this must be the foundation for defining the end goal of church planting. This question remains: Can church health be measured accurately?
To answer the question, it is important to understand what measurement is. Douglas Hubbard states that even in cases where the subject in question seems intangible and immeasurable, “If it’s something that you think exists at all, then it’s something you’ve already observed somehow” (2007, 267). It follows that if you are able to observe something, you are able to measure it in some way. Although all true measurements are expressed as a quantity, it is a mistake to assume that for something to be effectively measured, it needs to be reduced to an exact representative quantity. The point of measuring is not to eliminate all error, but to reduce uncertainty about the thing being observed so that we can make informed decisions.
Following this line of reasoning, the fact that we can talk about a church’s state of health shows that we’ve observed it, or the lack of it, in some way. It is now a question of being intentional in our method of measurement. In designing a measurement instrument for church planters, therefore, we must begin by defining what we want to observe, and then design a method for consistently quantifying our observations.
Is Your Organization Suffering from Measurement Inversion?
How do we know if we’re measuring the right things? The following exercise will help leaders identify whether their organization is suffering from measurement inversion.
1. Systematically work through reports from church planters and highlight activities and outcomes in two different colors. How much of the report has to do with activities and how much has to do with outcomes?
2. Are you measuring only what is easy (e.g., counting converts, baptisms, church attendance, etc.)? Or are you focusing on what is important (e.g., the spiritual health of emerging or immature churches)?
3. What do church planters do with reports? Are they simply filed and forgotten, or do they provide vital information that informs strategy?
If leaders are not satisfied with the answers to these questions, and conclude that they need to change what they measure, the following three steps will help them begin the process.
First, the leaders must count the cost of making this significant organizational change. When we shift from measuring activity to measuring outcomes, we are changing the rules of the game while the game is in progress. Despite leaders’ efforts to guide church planters through the transition, it is almost inevitable that some people will decide to leave the organization.
Second, the leaders must put together a design team. This should include one of the organization’s top executives, a project champion (or owner), and a person who is qualified to be the team’s theological watchdog. The design team will seek input from leaders and practitioners throughout the design process. Third, the project champion must be released from some current duties to focus on developing the instrument. The rest of this article is a step-by-step guide for the design team.
Define the Object of Measurement (i.e., Church)
As mentioned above, the end goal for church planters should be a biblically healthy church. The target “biblically healthy church,” however, is too vague to be observed and quantified. The design team needs to take the target and deconstruct it into sub-targets and observable indicators.
Sub-targets are the components that characterize a biblically healthy church. They don’t describe church-planting activities, but rather results of those activities. The list of sub-targets, when taken together, should be an accurate description of a healthy church without going beyond the biblical definition of church. The design team prepares a draft list of sub-targets, which they revise and rewrite as they receive input from leaders and practitioners. To keep the list of sub-targets manageable, the final list should be as short as possible (e.g., seven or fewer sub-targets).
The following questions can serve as a guide for the design team as they define sub-targets:
1. Is the sub-target a description of an outcome (rather than an activity)?
2. Is the sub-target an essential and irreducible component of a healthy church?
3. Taken together, do the identified sub-targets comprise an adequate description of a healthy church, or are other components still missing?
4. Does any sub-target go beyond what is biblical (i.e., do they reflect the organization’s traditions or cultural idiosyncrasies)?
One can think of sub-targets as a success checklist; when the condition is achieved, we check the corresponding box. Sub-targets, however, are not observable in a way that shows progress toward their achievement (Gohl 2003). For this reason, the design team needs to define at least one observable indicator for each sub-target.
An indicator is “the exemplary, concrete description of an essential feature of a sub-target” (Gohl 2003). A helpful starting point for defining indicators is to filter each sub-target through a series of questions and determine which answers identify its essential features. The key questions are: who, what, when, where, how, and how much/many? Not all six of these questions will be equally helpful for every sub-target. Focus only on features that provide information that affects decisions (Hubbard 2007, 96).
In other words, when these questions are posed, which answers affect or inform how a church-planting team would use its resources? Take, for example, the sub-target believers are discipled toward maturity. The question how could have many answers, but the actual methods, material, or program design are not essential since they focus on inputs rather than outcomes. The essential feature in this case is defined best by the question of what (i.e., we see believers increasingly demonstrate love for one another, spiritual hunger, the fruit of the Spirit, etc.).
Quantifying the Indicators
The design team should decide on the best method for scoring each indicator. When data are quantitative, scoring the indicator is relatively simple. For an indicator like “There are at least two elders,” you simply count. When quantitative data are expressed in homogenous units (e.g., euros), ratios can be calculated. For example, to measure the indicator “church operations are funded by local contributions,” compare total operational expenditures and income from local sources to determine what percentage of costs are covered by the church.
When indicators do not yield quantitative data, yet can be observed to varying degrees, it is best to rate the indicator on a scale. This is a bit trickier, since all qualitative measurements are subjective and depend on scorers’ judgments or opinions.
It is important, therefore, to design scales that provide consistent ratings from different raters. First, define what is meant by the each of the key terms. Second, design a rubric, a set of criteria that raters will use to determine how an item should be scored. The rubric in Figure 1 below is designed for the indicator, “There are biblically qualified elders.” It is based on the biblical qualifications for elders found in 1 Timothy 3:2-7. The rubric defines what a rater needs to observe in church elders in order to rate them accurately and consistently. Third, ensure that raters are trained to use the rubric.
Even with these guidelines, it often is difficult to decide which measurement method is best for each indicator. For example, should an indicator like “believers are discipled toward maturity” be measured on a scale, by a yes or no answer, or by calculating the percentage of church members who are actively being discipled? The following questions may help when deciding which measurement method should be used:
1. Which method best reduces uncertainty regarding the indicator’s essential feature?
2. Which method provides higher information value (i.e., it informs the decisions of the church-planting team)?
3. Which method is most likely to produce the same response when scored by different observers?
Avant Ministries’ Measurement Instrument
In 2002, Avant Ministries began to work through a process similar to the one described in this article. It was a two-year process in which leaders and church planters gave input to each draft of sub-targets and indicators. In retrospect, the leaders at Avant recognized that the process itself was valuable and ensured that the instrument would be sound both in theory and practice.
Additionally, it helped church planters buy-in to the new instrument. The following summary of Avant’s measurement instrument is, consequently, to be treated as an example of an outcomes-measurement instrument rather than an instrument to be adopted by other church planters.
Figure 2 below displays the deconstruction of the target church into five sub-targets and thirteen indicators. The thirteen indicators are scored using a 36-question survey that directs scorers to count, rate, or classify the indicators. The graph in Figure 3 below displays the overall achievement of the target and the relative strength of the thirteen indicators. Note that the graph does not dictate a ministry plan; rather, it gives a snapshot of the current health of the church plant. The team can then use their knowledge of the ministry context to decide how best to focus their efforts to strengthen the weak areas of the church.
I have worked on two church-planting projects with Avant Ministries. The two projects had a number of similarities. Both were set in predominantly Catholic Europe, where church planters were seeing very little fruit. My co-workers in both projects were gifted and passionate about seeing the church established. Yet one project was substantially more successful than the other. I believe it was largely due to the fact that in the successful project the team had a clear definition of church and an instrument for measuring progress toward the goal.
The clear goal and constant measurement kept us from straying from our mission, and guided our decision making. Most importantly, it gave us a glimpse into what God was doing, which caused us to take bigger steps of faith than we otherwise might have taken.
Breslin, Scott. 2007. “Church Planting Tracking and Analysis Tool.” Evangelical Missions Quarterly 43(3): 508-515.
Corwin, Gary. 2005. “Church Planting 101.” Evangelical Missions Quarterly 41(2): 142-143.
Deyneka Peter. 1999. Omega Course: Critical Church Planter Training: Manual Four. South Holland: Bible League.
European Union Joint Evaluation Unit. 2006. Evaluation Methods for the European Union’s External Assistance: Methodological Basis for Evaluation Vol. 1. Luxemburg: Office for Official Publications of the European Communities. Accessed February 1, 2013, from ec.europa.eu/europeaid/evaluation/methodology/examples/guide3_en.pdf.
Gohl, Eberhard. 2003. Checking and Learning: Impact Monitoring and Evaluation, a Practical Guide. The Association of German Development NGO’s (VENRO). Accessed February 1, 2013, from www.sle-berlin.de/sleplus/files Checking %20and%20learning.PDF.
Hubbard, Douglas. 2007. How to Measure Anything: Finding the Value of Intangibles in Business. Hoboken, N.J.: Wiley and Sons.
Walker, Philip 2005. “The Transition from Church Growth to Church Health.” Journal of the American Society for Church Growth 16: 3-13.
Warren, Rick. 1995. The Purpose Driven Church. Grand Rapids, Mich.: Zondervan.
Kenn Oke is European director for Avant Ministries. He and his wife, Doreen, have been with Avant since 1994, working as church planters in Spain and Poland. Kenn holds an MA in Intercultural Studies from Columbia International University.
EMQ, Vol. 49, No. 2, pp. 282-290. Copyright © 2013 Billy Graham Center. All rights reserved. Not to be reproduced or copied in any form without written permission from EMIS.